You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "tkalkirill (via GitHub)" <gi...@apache.org> on 2023/09/26 12:30:09 UTC
[GitHub] [ignite-3] tkalkirill opened a new pull request, #2631: IGNITE-20330 Create an abstraction for building indexes
tkalkirill opened a new pull request, #2631:
URL: https://github.com/apache/ignite-3/pull/2631
https://issues.apache.org/jira/browse/IGNITE-20330
Thank you for submitting the pull request.
To streamline the review process of the patch and ensure better code quality
we ask both an author and a reviewer to verify the following:
### The Review Checklist
- [ ] **Formal criteria:** TC status, codestyle, mandatory documentation. Also make sure to complete the following:
\- There is a single JIRA ticket related to the pull request.
\- The web-link to the pull request is attached to the JIRA ticket.
\- The JIRA ticket has the Patch Available state.
\- The description of the JIRA ticket explains WHAT was made, WHY and HOW.
\- The pull request title is treated as the final commit message. The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
- [ ] **Design:** new code conforms with the design principles of the components it is added to.
- [ ] **Patch quality:** patch cannot be split into smaller pieces, its size must be reasonable.
- [ ] **Code quality:** code is clean and readable, necessary developer documentation is added if needed.
- [ ] **Tests code quality:** test set covers positive/negative scenarios, happy/edge cases. Tests are effective in terms of execution time and resources.
### Notes
- [Apache Ignite Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348401853
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> I don't think it's specified, but it is what we use commonly. It's like having the size in bytes by default, but then creating long FOO_SIZE = 10 meaning 10 kilobytes, it's confusing.
Commonly where? There are different options for the project, it may be more correct to add a unit of measurement of _MIILS, _SEC and etc?
> It was a question. If we wait indefinitely, then why do we pass a 10 seconds timeout instead of passing an infinite timeout? This is literally what I asked, there was no proposal.
Now I understand, I’ll try to wait forever.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> I don't think it's specified, but it is what we use commonly. It's like having the size in bytes by default, but then creating long FOO_SIZE = 10 meaning 10 kilobytes, it's confusing.
Commonly where? There are different options for the project, it may be more correct to add a unit of measurement of _MIILS, _SEC and etc?
> It was a question. If we wait indefinitely, then why do we pass a 10 seconds timeout instead of passing an infinite timeout? This is literally what I asked, there was no proposal.
Now I understand, I’ll try to wait forever.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348811659
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348629559
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItDataSchemaSyncTest.java:
##########
@@ -62,14 +60,9 @@
*/
@ExtendWith(WorkDirectoryExtension.class)
public class ItDataSchemaSyncTest extends IgniteAbstractTest {
- /**
- * Table name.
- */
public static final String TABLE_NAME = "tbl1";
- /**
- * Nodes bootstrap configuration.
- */
+ /** Nodes bootstrap configuration. */
Review Comment:
We discussed it as a team and decided not to remove the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347365459
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
Review Comment:
"expired", for example. This word is used in the project for the same purpose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347354767
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -308,6 +311,12 @@ public class IgniteImpl implements Ignite {
/** System views manager. */
private final SystemViewManagerImpl systemViewManager;
+ /** Index builder. */
+ private final IndexBuilder indexBuilder;
Review Comment:
Ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347292196
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
Review Comment:
What's your suggestion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348368810
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
I will explain my issue with the name. While reading the code, I assumed that passed closure will be executed in different thread. This changes the perception of the code completely. You start thinking that the operation is async, which implies different properties that you check. Why did you called a synchronous operation "Async", that's beyond me
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
I will explain my issue with the name. While reading the code, I assumed that passed closure will be executed in different thread. This changes the perception of the code completely. You start thinking that the operation is async, which implies different properties that you check. Why did you call a synchronous operation "Async", that's beyond me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348369584
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
Which postfix do you think is better?
1. IfPrimacyExpire
2. IfPrimaryExpire
> Anyway, why do we need a "if primacy lost", is the reason important to perform the operation?
I did it by analogy with method `java.util.Map#putIfAbsent`, which hints that the operation will be performed only if the primary replica expired. First of all, it is convenient for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348363196
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
I understand now, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348604649
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
Here's what's possible: after receiving the meta, at the moment when we want to add a task, the replica will be expired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348627579
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -19,11 +19,8 @@
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
-import org.apache.ignite.network.ClusterNode;
-/**
- * Replica listener.
- */
+/** Replica listener. */
Review Comment:
We discussed it as a team and decided not to remove the changes.
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -35,18 +32,7 @@ public interface ReplicaListener {
*/
CompletableFuture<?> invoke(ReplicaRequest request, String senderId);
- /**
- * Callback on becoming the primary replica.
- *
- * @param clusterNode Primary replica node.
- */
- default void onBecomePrimary(ClusterNode clusterNode) {
- // No-op.
- }
-
- /**
- * Callback on replica shutdown.
- */
+ /** Callback on replica shutdown. */
Review Comment:
We discussed it as a team and decided not to remove the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348468107
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
> I can't find a convention that states that the code that returns the future must be executed in another thread.
There's no such convention. Never was. You are misinterpreting what I said.
I say that method, that has `Async` suffix, does execute the operation asynchronously. For example, `java.util.concurrent.CompletableFuture#runAsync(java.lang.Runnable)` runs a closure in a separate thread.
A better example is a pair `thenCompose` and `thenComposeAsync`. Both these methods acept closure that returns the future, just like in your case. `thenCompose` executes the closure in the same thread, and `thenComposeAsync` executes closure in a threadpool.
In your case, you execute the closure in the same thread, but call the method `Async`. This is counter-intuitive.
> Could you provide information where this is stated?
I don't get it, what do I need to provide. This is what all people do without a formal written agreement, as far as I'm concerned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348623620
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
+ CompletableFuture<ReplicaMeta> replicaMetaFuture = completedFuture(replicaMetaForOneSecond(leaseholder, startTime));
+
+ assertThat(placementDriver.setPrimaryReplicaMeta(0, replicaId(partitionId), replicaMetaFuture), willCompleteSuccessfully());
+ }
+
+ private int tableId() {
+ return getTableIdStrict(catalogManager, TABLE_NAME, clock.nowLong());
+ }
+
+ private int indexId(String indexName) {
+ return getIndexIdStrict(catalogManager, indexName, clock.nowLong());
+ }
+
+ private TablePartitionId replicaId(int partitionId) {
+ return new TablePartitionId(tableId(), partitionId);
+ }
+
+ private static TestReplicaMeta replicaMetaForOneSecond(String leaseholder, HybridTimestamp startTime) {
+ HybridTimestamp expirationTime = new HybridTimestamp(startTime.getPhysical() + 1_000, startTime.getLogical());
Review Comment:
fix it
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
+ CompletableFuture<ReplicaMeta> replicaMetaFuture = completedFuture(replicaMetaForOneSecond(leaseholder, startTime));
+
+ assertThat(placementDriver.setPrimaryReplicaMeta(0, replicaId(partitionId), replicaMetaFuture), willCompleteSuccessfully());
+ }
+
+ private int tableId() {
+ return getTableIdStrict(catalogManager, TABLE_NAME, clock.nowLong());
+ }
+
+ private int indexId(String indexName) {
+ return getIndexIdStrict(catalogManager, indexName, clock.nowLong());
+ }
+
+ private TablePartitionId replicaId(int partitionId) {
+ return new TablePartitionId(tableId(), partitionId);
+ }
+
+ private static TestReplicaMeta replicaMetaForOneSecond(String leaseholder, HybridTimestamp startTime) {
+ HybridTimestamp expirationTime = new HybridTimestamp(startTime.getPhysical() + 1_000, startTime.getLogical());
+
+ return new TestReplicaMeta(leaseholder, startTime, expirationTime);
+ }
+
+ private IndexBuildController newIndexBuildController() {
+ return new IndexBuildController(indexBuilder, indexManager, catalogManager, clusterService, placementDriver, clock);
+ }
+
+ private static class TestPlacementDriver extends AbstractEventProducer<PrimaryReplicaEvent, PrimaryReplicaEventParameters> implements
+ PlacementDriver {
+ private final Map<ReplicationGroupId, CompletableFuture<ReplicaMeta>> primaryReplicaMetaFutureById = new ConcurrentHashMap<>();
+
+ @Override
+ public CompletableFuture<ReplicaMeta> awaitPrimaryReplica(
+ ReplicationGroupId groupId,
+ HybridTimestamp timestamp,
+ long timeout,
+ TimeUnit unit
+ ) {
+ return primaryReplicaMetaFutureById.get(groupId);
+ }
+
+ @Override
+ public CompletableFuture<ReplicaMeta> getPrimaryReplica(ReplicationGroupId replicationGroupId, HybridTimestamp timestamp) {
+ return primaryReplicaMetaFutureById.get(replicationGroupId);
+ }
+
+ @Override
+ public CompletableFuture<Void> previousPrimaryExpired(ReplicationGroupId grpId) {
+ throw new UnsupportedOperationException();
+ }
+
+ CompletableFuture<Void> setPrimaryReplicaMeta(
+ long causalityToken,
+ TablePartitionId replicaId,
+ CompletableFuture<ReplicaMeta> replicaMetaFuture
+ ) {
+ primaryReplicaMetaFutureById.put(replicaId, replicaMetaFuture);
+
+ return replicaMetaFuture.thenCompose(replicaMeta -> fireEvent(
+ PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED,
+ new PrimaryReplicaEventParameters(causalityToken, replicaId, replicaMeta.getLeaseholder())
+ ));
+ }
+ }
+
+ private static class TestReplicaMeta implements ReplicaMeta {
Review Comment:
Fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348626965
##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/PlacementDriverManager.java:
##########
@@ -258,16 +245,6 @@ private void recoverInternalComponentsBusy() {
long recoveryRevision = recoveryFinishedFuture.join();
- CompletableFuture<Void> startLeaserTrackerFuture = leaseTracker.startTrackAsync(recoveryRevision);
-
- // Forces to wait until recovery is complete before the metastore watches is deployed to avoid races with other components.
- startVv.update(recoveryRevision, (unused, throwable) -> startLeaserTrackerFuture)
- .whenComplete((unused, throwable) -> {
- if (throwable != null) {
- LOG.error("Error starting the PlacementDriverManager internal components", throwable);
- } else {
- LOG.debug("Internal components of the PlacementDriverManager started successfully");
- }
- });
+ leaseTracker.startTrackAsync(recoveryRevision);
Review Comment:
Fix naming
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347366303
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I would ask authors of that constant, honestly. Seems arbitrary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347339136
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
I haven’t yet understood why this is not consistent and how to fix it, please explain.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
I haven’t yet understood why this is not consistent and how to fix it, please explain.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348373958
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
Since these events are processed by the metastore, then yes they are sequential.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348411407
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
I can't find a convention that states that the code that returns the future must be executed in another thread.
Could you provide information where this is stated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347356253
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
What do you offer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348601859
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347300412
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
Review Comment:
Is it necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347349441
##########
modules/placement-driver-api/src/main/java/org/apache/ignite/internal/placementdriver/event/PrimaryReplicaEvent.java:
##########
@@ -33,7 +33,7 @@ public enum PrimaryReplicaEvent implements Event {
* <p>Notes:</p>
* <ul>
* <li>This event will fire strictly after the completion of the future from {@link PlacementDriver#awaitPrimaryReplica}.</li>
- * <li>This event will fire when on node recovery and will indicate the primary replica at the time the node was stopped.</li>
+ * <li>This event will fire in the metastore thread.</li>
Review Comment:
Yep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347354068
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -19,11 +19,8 @@
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
-import org.apache.ignite.network.ClusterNode;
-/**
- * Replica listener.
- */
+/** Replica listener. */
Review Comment:
Please show the rule where this is prohibited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348344616
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I suggest reusing the same constant in that case, assuming that "in the future it will change", but it's up to you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348350426
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
I'm talking about "awaitPrimaryReplicaFowNow", I thought you can tell that from the context.
What kind of simple operations are you talking about? Scanning through partitions and reading last built row from them? This is IO, it may not be as fast as you hope. Or you mean other operations?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348384877
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
What exception is thrown from the `node.stop()`? My proposal is not to throw random `IgniteException` instances in violation of existing error handling rules. Maybe we need a separate issue for the fix, I would accept that.
Are you familiar with the way we treat public exceptions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348609013
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
Review Comment:
Fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348799968
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
+ }
+ }
+
+ return completedFuture(replicaMeta);
+ }).thenCompose(Function.identity());
+ }
+
+ private void startBuildIndex(TablePartitionId replicaId, CatalogIndexDescriptor indexDescriptor, MvTableStorage mvTableStorage) {
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348796286
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
We discussed it in person, adding the postfix `_SEC`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348801577
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
Fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347362953
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
I propose stopping these two components the same way we stop every other component
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348389893
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
Why is there suddenly a new exception from the stop method? Is this because of new managers that you close? I propose changing that code instead, and reverting this particular change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348609316
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347291209
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
What kind of operation are you talking about? Physical building of the index?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348445203
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
Ok, I was confused by what "startBuild" does. I guess I should rely more on the documentation of methods, rather than their names
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348616137
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348441269
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
> we simply add a task to build the index
Ok, if that's all that we do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348430649
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
It’s not entirely clear to me what the criteria must be for a class to become a `IgniteComponent`, can you explain?
If the answer is: to use it in `LifecycleManager`, that doesn't seem like a reasonable reason.
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
It’s not entirely clear to me what the criteria must be for a class to become a `IgniteComponent`, can you explain?
If the answer is: to use it in `LifecycleManager`, that doesn't seem like a reasonable reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1350089135
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -185,62 +191,70 @@ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParamete
int catalogVersion = catalogService.latestCatalogVersion();
return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
- .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
- .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaForNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryScheduleBuildIndexesForNewPrimaryReplica(
catalogVersion,
primaryReplicaId,
mvTableStorage,
replicaMeta
))
);
} else {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return completedFuture(null);
}
});
}
- private void tryStartBuildIndexesForNewPrimaryReplica(
+ private void tryScheduleBuildIndexesForNewPrimaryReplica(
int catalogVersion,
TablePartitionId primaryReplicaId,
MvTableStorage mvTableStorage,
ReplicaMeta replicaMeta
) {
inBusyLock(busyLock, () -> {
if (isLeaseExpire(replicaMeta)) {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return;
}
// TODO: IGNITE-20530 We only need to get write-only indexes
for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
- startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ scheduleBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
}
}
});
}
- private void tryStartBuildIndex(
+ private void tryScheduleBuildIndex(
TablePartitionId primaryReplicaId,
CatalogIndexDescriptor indexDescriptor,
MvTableStorage mvTableStorage,
ReplicaMeta replicaMeta
) {
inBusyLock(busyLock, () -> {
if (isLeaseExpire(replicaMeta)) {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return;
}
- startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ scheduleBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
});
}
- private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ /**
+ * Stops building indexes for a replica that has expired, if it has not been done before.
+ *
+ * <p>We need to stop building indexes at the event of a change of primary replica or after executing asynchronous code, we understand
+ * that the {@link ReplicaMeta#getExpirationTime() expiration time} has come.</p>
+ *
+ * @param replicaId Replica ID.
+ */
+ private void stopBuildingIndexesIfIfPrimaryExpired(TablePartitionId replicaId) {
Review Comment:
LOL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348371251
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
This test asserts that method "startBuildIndex" is executed synchronously inside of the "createIndex" method. My question is - are we sure, that it must be executed synchronously inside of the "createIndex" method? Why is it necessary to do this way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348603345
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348802435
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
Fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348835603
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
improve it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348436782
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
> What exception is thrown from the `node.stop()`?
java.lang.Exception
> Are you familiar with the way we treat public exceptions?
No.
> Why is there suddenly a new exception from the stop method?
I think the previous code was wrong
` public void stop() {
lifecycleManager.stopNode();
restAddressReporter.removeReport();
}`
if we had an error from `stopNode` then we would not execute `removeReport`, which does not seem correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348448015
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
It's more like "removeIfPresent", if I were to use your analogy. I would use `IfPrimaryExpire**d**`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348454046
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> Commonly where?
In the project. If there are other constants, that mean seconds without being named _SEC or something, they are just as confusing.
Again, I don't understand why we're arguing. Do you assume, that for every constant we should look for its usages to understand what it means? That would be an absurd claim, if we're not in tests.
I also understand that you copied the name without second thought. But the moment you did that, you became the author of the code. If you think that adding _SEC or _SECONDS will make the code worse, please explain why
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347299773
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
I don’t quite understand what’s the typo here? There is such a word, here is the link: https://dictionary.cambridge.org/dictionary/english/primacy
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
I don’t quite understand what’s the typo here? There is such a word, here is the link: https://dictionary.cambridge.org/dictionary/english/primacy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347289059
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
Review Comment:
If we simply return the completed future, then there may be races with the operations of selecting a primary replica and deleting the index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347331771
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
Yes, this is possible, for example, a new replica appeared and wanted to build indexes for all indexes, and while the storage was being received, the replica expired, then for each index you will have to stop building indexes for this replica.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348357052
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
Review Comment:
Ok. For the future, I recommend less generic names. We already have too many instances of "table/getTable" with completely different semantics. I don't want to continue that trend. Thank you for documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348370516
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
Review Comment:
I'll leave it as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347284882
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
Review Comment:
It cannot, because the result of selecting the primary replica is stored in the metastore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347336084
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> So the timeout constant is in seconds. Please reflect that in the name. By default all timeouts are defined in milliseconds, so this one leads to confusion
I don’t know about this, where can I look about this rule?
> If we wait indefinitely, then why do we split in into 10-seconds chunks?
As far as I can see, this is what is happening now, or I didn’t understand your proposal.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> So the timeout constant is in seconds. Please reflect that in the name. By default all timeouts are defined in milliseconds, so this one leads to confusion
I don’t know about this, where can I look about this rule?
> If we wait indefinitely, then why do we split in into 10-seconds chunks?
As far as I can see, this is what is happening now, or I didn’t understand your proposal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347347118
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
If you are talking about `inBusyLockAsync`, it is called that because it returns the future, this is described in its documentation.
If you think it is worth improving the name or description of method `inBusyLockAsync`, then please create a ticket for this.
> Why do we even have a specific version for completable future
Because there are `inBusyLock` methods with the same semantics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347453831
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I checked with @sanpwc, he says that at the moment for `PartitionReplicaListener#AWAIT_PRIMARY_REPLICA_TIMEOUT` is is valid value, in the future it will change.
I think in my case this is a reasonable time, if you don’t think so, then suggest which would be better.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I checked with @sanpwc, he says that at the moment for `PartitionReplicaListener#AWAIT_PRIMARY_REPLICA_TIMEOUT` is is valid value, in the future it will change.
I think in my case this is a reasonable time, if you don’t think so, then suggest which would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347296951
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
What future are you talking about?
In general, this is not so important; simple operations are performed here; the physical building of indexes is performed in the `org.apache.ignite.internal.table.distributed.index.IndexBuilder#executor`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348375482
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
Review Comment:
I understand your point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348425121
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
There is not a word in method `org.mockito.Mockito#verify(T)` that it checks the method call in a specific thread.
I’m just checking that this method will be executed as a result of the index creation operation, without being tied to a thread.
What's ultimately wrong with this?
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
There is not a word in method `org.mockito.Mockito#verify(T)` that it checks the method call in a specific thread.
I’m just checking that this method will be executed as a result of the index creation operation, without being tied to a thread.
What's ultimately wrong with this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348357052
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
Review Comment:
Ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348348594
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
I thought you copied the name from another method. If all these methods are yours, then my assumption is wrong. Yes, I meant the word "fow"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill merged PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348801069
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
We discussed it personally until we found a better name for `inBusyLockAsync` so we’ll leave it as is.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
We discussed it personally until we found a better name for `inBusyLockAsync` so we’ll leave it as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348803769
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
We discussed the issue, the issue was removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348352871
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
So you meant "primacy", not "primary". Ok, my bad in that case. I assumed that because there are no usages of the word "primacy" in the project.
Standard terminology would fit better in my opinion.
Anyway, why do we need a "if primacy lost", is the reason important to perform the operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348345041
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
Review Comment:
Ok, I see
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348450726
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
You can rename it to `scheduleBuildIndex` (maybe sounds like something to plan for later) or `addBuildIndexTask`, what do you think?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
We can rename it to `scheduleBuildIndex` (maybe sounds like something to plan for later) or `addBuildIndexTask`, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348616580
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1346941969
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
Review Comment:
Is this really necessary? What will happen if we return a completed future here?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
Please specify, why we need it and why it's 10
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
Review Comment:
Can primary replica be reelected concurrently? I guess not, but asking just in case
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
Can this be false? How?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
In case of success we return a future, but in case of exception we throw it. This is inconsistent. Can you please fix it, or explain why it should be this way?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
`FowNow`. Maybe that's not your typo, but I suggest fixing it here anyway. There shouldn't be too many usages.
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
`10` should be a constant as well
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
+ CompletableFuture<ReplicaMeta> replicaMetaFuture = completedFuture(replicaMetaForOneSecond(leaseholder, startTime));
+
+ assertThat(placementDriver.setPrimaryReplicaMeta(0, replicaId(partitionId), replicaMetaFuture), willCompleteSuccessfully());
+ }
+
+ private int tableId() {
+ return getTableIdStrict(catalogManager, TABLE_NAME, clock.nowLong());
+ }
+
+ private int indexId(String indexName) {
+ return getIndexIdStrict(catalogManager, indexName, clock.nowLong());
+ }
+
+ private TablePartitionId replicaId(int partitionId) {
+ return new TablePartitionId(tableId(), partitionId);
+ }
+
+ private static TestReplicaMeta replicaMetaForOneSecond(String leaseholder, HybridTimestamp startTime) {
+ HybridTimestamp expirationTime = new HybridTimestamp(startTime.getPhysical() + 1_000, startTime.getLogical());
Review Comment:
There's a `org.apache.ignite.internal.hlc.HybridTimestamp#addPhysicalTime` that can be used for it.
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
Review Comment:
Please create an utility method somewhere to generate the name of PK index. There are, like, 3 copies of this algorithm in the code, would be nice to have only one
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -35,18 +32,7 @@ public interface ReplicaListener {
*/
CompletableFuture<?> invoke(ReplicaRequest request, String senderId);
- /**
- * Callback on becoming the primary replica.
- *
- * @param clusterNode Primary replica node.
- */
- default void onBecomePrimary(ClusterNode clusterNode) {
- // No-op.
- }
-
- /**
- * Callback on replica shutdown.
- */
+ /** Callback on replica shutdown. */
Review Comment:
Same here
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
Is this a good exception for the purpose? This is a public API, and according to the error handling design, all public error codes should be associated with the specific implementation class. Ideally, it also should be documented in the method that throws the exception. I don't see any of that.
What was the reason here?
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -578,6 +586,17 @@ public class IgniteImpl implements Ignite {
indexManager = new IndexManager(schemaManager, distributedTblMgr, catalogManager, metaStorageMgr, registry);
+ indexBuilder = new IndexBuilder(name, Runtime.getRuntime().availableProcessors(), replicaSvc);
+
+ indexBuildController = new IndexBuildController(
+ indexBuilder,
Review Comment:
Ok, I guess no one, except for the controller, uses index builder. But, we accept the builder as a dependency, because it simplifies tests?
On one hand, I'd love to get rid of the constructor parameter, but on the other, your approach is more flexible. This comment is just my thoughts, you don't need to change anything
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replication/request/BuildIndexReplicaRequest.java:
##########
@@ -19,11 +19,16 @@
import java.util.List;
import java.util.UUID;
+import org.apache.ignite.internal.replicator.exception.PrimaryReplicaMissException;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
import org.apache.ignite.internal.table.distributed.TableMessageGroup;
import org.apache.ignite.network.annotations.Transferable;
-/** Replica request to build a table index. */
+/**
+ * Replica request to build a table index.
+ *
+ * <p>It is possible to receive an {@link PrimaryReplicaMissException} in response to message processing if the leaseholder changes.</p>
Review Comment:
```suggestion
* <p>It is possible to receive a {@link PrimaryReplicaMissException} in response to message processing if the leaseholder changes.</p>
```
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
What thread completes that future? Is this important?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
From the naming, it's hard to understand, what thread will execute the operation. There's no documentation either, and code is not trivial
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java:
##########
@@ -103,18 +104,18 @@ public class IndexManager implements IgniteComponent {
*
* @param schemaManager Schema manager.
* @param tableManager Table manager.
- * @param catalogManager Catalog manager.
+ * @param catalogService Catalog manager.
*/
public IndexManager(
CatalogSchemaManager schemaManager,
TableManager tableManager,
- CatalogManager catalogManager,
+ CatalogManager catalogService,
Review Comment:
Excuse me... but it is a manager
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
Review Comment:
Can we move this to a javadoc?
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
Review Comment:
Should we stop everything else? Or there are no stop methods in other components?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
Review Comment:
It's not lost, please use another word
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
Review Comment:
`Primacy`, another typo. This one must be yours
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
So the timeout constant is in seconds. Please reflect that in the name. By default all timeouts are defined in milliseconds, so this one leads to confusion
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
Review Comment:
Not very descriptive. Looks like this method returns new instance and doesn't have any side effects, which is probably not true. If it is true, then what's the point? Very confusing.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
Review Comment:
This is a bad naming for the method. It only mentions the return type, but not the purpose. It's hard to come up with a good suggestion, but the idea is to communicate the "get table with created indexes" semantics that this method has
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
Review Comment:
This name is sooooo confusing. Why is it called async, if it's literally executed in the same thread? Please fix it.
Why do we even have a specific version for completable future, if it's just another object? I'm asking you because you seem to be an author.
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
Are we sure that we want this operation in the same thread?
##########
modules/placement-driver-api/src/testFixtures/java/org/apache/ignite/internal/placementdriver/TestPlacementDriver.java:
##########
@@ -69,6 +69,6 @@ public void removeListener(PrimaryReplicaEvent evt, EventListener<? extends Prim
@Override
public CompletableFuture<Void> previousPrimaryExpired(ReplicationGroupId grpId) {
- return CompletableFuture.completedFuture(null);
+ return completedFuture(null);
Review Comment:
Interesting. Import is not updated. Why did you do it? :)
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
Review Comment:
It's not clear what you mean by "for one second". Can you please elaborate in the comment?
##########
modules/placement-driver-api/src/main/java/org/apache/ignite/internal/placementdriver/event/PrimaryReplicaEvent.java:
##########
@@ -33,7 +33,7 @@ public enum PrimaryReplicaEvent implements Event {
* <p>Notes:</p>
* <ul>
* <li>This event will fire strictly after the completion of the future from {@link PlacementDriver#awaitPrimaryReplica}.</li>
- * <li>This event will fire when on node recovery and will indicate the primary replica at the time the node was stopped.</li>
+ * <li>This event will fire in the metastore thread.</li>
Review Comment:
So the old statement is not correct, right?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
+ }
+ }
+
+ return completedFuture(replicaMeta);
+ }).thenCompose(Function.identity());
+ }
+
+ private void startBuildIndex(TablePartitionId replicaId, CatalogIndexDescriptor indexDescriptor, MvTableStorage mvTableStorage) {
Review Comment:
Please add a javadoc. This entire class barely has any documentation
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
+ CompletableFuture<ReplicaMeta> replicaMetaFuture = completedFuture(replicaMetaForOneSecond(leaseholder, startTime));
+
+ assertThat(placementDriver.setPrimaryReplicaMeta(0, replicaId(partitionId), replicaMetaFuture), willCompleteSuccessfully());
+ }
+
+ private int tableId() {
+ return getTableIdStrict(catalogManager, TABLE_NAME, clock.nowLong());
+ }
+
+ private int indexId(String indexName) {
+ return getIndexIdStrict(catalogManager, indexName, clock.nowLong());
+ }
+
+ private TablePartitionId replicaId(int partitionId) {
+ return new TablePartitionId(tableId(), partitionId);
+ }
+
+ private static TestReplicaMeta replicaMetaForOneSecond(String leaseholder, HybridTimestamp startTime) {
+ HybridTimestamp expirationTime = new HybridTimestamp(startTime.getPhysical() + 1_000, startTime.getLogical());
+
+ return new TestReplicaMeta(leaseholder, startTime, expirationTime);
+ }
+
+ private IndexBuildController newIndexBuildController() {
+ return new IndexBuildController(indexBuilder, indexManager, catalogManager, clusterService, placementDriver, clock);
+ }
+
+ private static class TestPlacementDriver extends AbstractEventProducer<PrimaryReplicaEvent, PrimaryReplicaEventParameters> implements
+ PlacementDriver {
+ private final Map<ReplicationGroupId, CompletableFuture<ReplicaMeta>> primaryReplicaMetaFutureById = new ConcurrentHashMap<>();
+
+ @Override
+ public CompletableFuture<ReplicaMeta> awaitPrimaryReplica(
+ ReplicationGroupId groupId,
+ HybridTimestamp timestamp,
+ long timeout,
+ TimeUnit unit
+ ) {
+ return primaryReplicaMetaFutureById.get(groupId);
+ }
+
+ @Override
+ public CompletableFuture<ReplicaMeta> getPrimaryReplica(ReplicationGroupId replicationGroupId, HybridTimestamp timestamp) {
+ return primaryReplicaMetaFutureById.get(replicationGroupId);
+ }
+
+ @Override
+ public CompletableFuture<Void> previousPrimaryExpired(ReplicationGroupId grpId) {
+ throw new UnsupportedOperationException();
+ }
+
+ CompletableFuture<Void> setPrimaryReplicaMeta(
+ long causalityToken,
+ TablePartitionId replicaId,
+ CompletableFuture<ReplicaMeta> replicaMetaFuture
+ ) {
+ primaryReplicaMetaFutureById.put(replicaId, replicaMetaFuture);
+
+ return replicaMetaFuture.thenCompose(replicaMeta -> fireEvent(
+ PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED,
+ new PrimaryReplicaEventParameters(causalityToken, replicaId, replicaMeta.getLeaseholder())
+ ));
+ }
+ }
+
+ private static class TestReplicaMeta implements ReplicaMeta {
Review Comment:
Would it be simpler to use `Lease` class instead of the test implementation?
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -19,11 +19,8 @@
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
-import org.apache.ignite.network.ClusterNode;
-/**
- * Replica listener.
- */
+/** Replica listener. */
Review Comment:
Why did you reformat completely valid comment? Please stop making stylistic changes in unrelated pieces of code.
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItDataSchemaSyncTest.java:
##########
@@ -62,14 +60,9 @@
*/
@ExtendWith(WorkDirectoryExtension.class)
public class ItDataSchemaSyncTest extends IgniteAbstractTest {
- /**
- * Table name.
- */
public static final String TABLE_NAME = "tbl1";
- /**
- * Nodes bootstrap configuration.
- */
+ /** Nodes bootstrap configuration. */
Review Comment:
You removed some comments, reformatted others. Why?
##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/PlacementDriverManager.java:
##########
@@ -258,16 +245,6 @@ private void recoverInternalComponentsBusy() {
long recoveryRevision = recoveryFinishedFuture.join();
- CompletableFuture<Void> startLeaserTrackerFuture = leaseTracker.startTrackAsync(recoveryRevision);
-
- // Forces to wait until recovery is complete before the metastore watches is deployed to avoid races with other components.
- startVv.update(recoveryRevision, (unused, throwable) -> startLeaserTrackerFuture)
- .whenComplete((unused, throwable) -> {
- if (throwable != null) {
- LOG.error("Error starting the PlacementDriverManager internal components", throwable);
- } else {
- LOG.debug("Internal components of the PlacementDriverManager started successfully");
- }
- });
+ leaseTracker.startTrackAsync(recoveryRevision);
Review Comment:
This change is not obvious to me. Can you please comment on it?
We also need a review from the "class owner"
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -308,6 +311,12 @@ public class IgniteImpl implements Ignite {
/** System views manager. */
private final SystemViewManagerImpl systemViewManager;
+ /** Index builder. */
+ private final IndexBuilder indexBuilder;
Review Comment:
I think it would be nice to encapsulate some of these components into another umbrella component. IgniteImpl becomes too big. Not this time, of course
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replication/request/BuildIndexReplicaRequest.java:
##########
@@ -19,11 +19,16 @@
import java.util.List;
import java.util.UUID;
+import org.apache.ignite.internal.replicator.exception.PrimaryReplicaMissException;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
import org.apache.ignite.internal.table.distributed.TableMessageGroup;
import org.apache.ignite.network.annotations.Transferable;
-/** Replica request to build a table index. */
+/**
+ * Replica request to build a table index.
+ *
+ * <p>It is possible to receive an {@link PrimaryReplicaMissException} in response to message processing if the leaseholder changes.</p>
Review Comment:
Is this addition really relevant? I mean, you can receive that exception for all replica requests, right? I don't understand the placement of this comment in this particular class.
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
Review Comment:
By the way, I like how clean these tests look, thank you!
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
I don't think that this is how it's supposed to be. Shouldn't "lifecycleManager" close everything? I don't see all the other components in this list, which means that you most likely did something wrong.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
If we wait indefinitely, then why do we split in into 10-seconds chunks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347347885
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
Review Comment:
I did not understand the question.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347354449
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItDataSchemaSyncTest.java:
##########
@@ -62,14 +60,9 @@
*/
@ExtendWith(WorkDirectoryExtension.class)
public class ItDataSchemaSyncTest extends IgniteAbstractTest {
- /**
- * Table name.
- */
public static final String TABLE_NAME = "tbl1";
- /**
- * Nodes bootstrap configuration.
- */
+ /** Nodes bootstrap configuration. */
Review Comment:
Please show the rule where this is prohibited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347304285
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
Review Comment:
This method was created in another ticket, and it has a description that you can read.
If you think it's worth changing it, then please create a ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347278842
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I don’t know what exactly to specify here, I took the value by analogy with `org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#AWAIT_PRIMARY_REPLICA_TIMEOUT`, do you have a suggestion for what to specify here?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I don’t know what exactly to specify here, I took the value by analogy with `org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#AWAIT_PRIMARY_REPLICA_TIMEOUT`, do you have a suggestion for what to specify here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348362509
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> I don’t know about this, where can I look about this rule?
I don't think it's specified, but it is what we use commonly. It's like having the size in bytes by default, but then creating `long FOO_SIZE = 10` meaning 10 kilobytes, it's confusing.
> Or I didn’t understand your proposal.
It was a question. If we wait indefinitely, then why do we pass a 10 seconds timeout instead of passing an infinite timeout? This is literally what I asked, there was no proposal.
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
Review Comment:
> I don’t know about this, where can I look about this rule?
I don't think it's specified, but it is what we use commonly. It's like having the size in bytes by default, but then creating `long FOO_SIZE = 10` meaning 10 kilobytes, it's confusing.
> Or I didn’t understand your proposal.
It was a question. If we wait indefinitely, then why do we pass a 10 seconds timeout instead of passing an infinite timeout? This is literally what I asked, there was no proposal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348363894
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
Dude! I literally explained the problem in the first statement. Why do you throw an exception instead of returning exceptionally competed future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348361088
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
Review Comment:
I'll leave it as it is for now, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348605342
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java:
##########
@@ -103,18 +104,18 @@ public class IndexManager implements IgniteComponent {
*
* @param schemaManager Schema manager.
* @param tableManager Table manager.
- * @param catalogManager Catalog manager.
+ * @param catalogService Catalog manager.
*/
public IndexManager(
CatalogSchemaManager schemaManager,
TableManager tableManager,
- CatalogManager catalogManager,
+ CatalogManager catalogService,
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348626407
##########
modules/placement-driver-api/src/testFixtures/java/org/apache/ignite/internal/placementdriver/TestPlacementDriver.java:
##########
@@ -69,6 +69,6 @@ public void removeListener(PrimaryReplicaEvent evt, EventListener<? extends Prim
@Override
public CompletableFuture<Void> previousPrimaryExpired(ReplicationGroupId grpId) {
- return CompletableFuture.completedFuture(null);
+ return completedFuture(null);
Review Comment:
For beauty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347294217
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
I don’t quite understand, but I made a typo in `FowNow`, should there be `ForNow` or did you mean something else?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
Review Comment:
I don’t quite understand, but I made a typo in `FowNow`, should there be `ForNow` or did you mean something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347365640
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
From the `org.apache.ignite.internal.app.LifecycleManager` documentation it is only about `IgniteComponent`, not about other types of components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348353509
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
Review Comment:
It looks like a comment for the method itself, not a line of code. Of course it's up to you, but I find this decision a bit strange
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348356644
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
Aren't these operations (election and expiration) sequential? This is not intuitive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348388150
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
Review Comment:
Please look at implementation `org.apache.ignite.internal.table.distributed.index.IndexBuilder#startBuildIndex`, essentially we are just putting the task of building the index in a separate thread pool. I can point this out in documentation of `IndexBuilder#startBuildIndex`, or what do you suggest?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348802134
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/replication/request/BuildIndexReplicaRequest.java:
##########
@@ -19,11 +19,16 @@
import java.util.List;
import java.util.UUID;
+import org.apache.ignite.internal.replicator.exception.PrimaryReplicaMissException;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
import org.apache.ignite.internal.table.distributed.TableMessageGroup;
import org.apache.ignite.network.annotations.Transferable;
-/** Replica request to build a table index. */
+/**
+ * Replica request to build a table index.
+ *
+ * <p>It is possible to receive an {@link PrimaryReplicaMissException} in response to message processing if the leaseholder changes.</p>
Review Comment:
I corrected the typo and discussed it personally and decided to leave it as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348376014
##########
modules/replicator/src/main/java/org/apache/ignite/internal/replicator/listener/ReplicaListener.java:
##########
@@ -19,11 +19,8 @@
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.replicator.message.ReplicaRequest;
-import org.apache.ignite.network.ClusterNode;
-/**
- * Replica listener.
- */
+/** Replica listener. */
Review Comment:
The rule is simple - common sense. We have a piece of code, that's completely fine and follows all the rules that we have. It has nothing to do with your PR. There's no rule that it should look differently. Yet you change it.
This is a useless change. You're simply enforcing your own stylistic choice. I thought that we don't enforce our own stylistic choices. I hope that's clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348383839
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
We block the metastore thread for as long as necessary, we wait for several operations that must be completed in the current revision (creating index storage and obtaining a primary replica). After that, we simply add a task to build the index, this is not a long and complicated operation, I don’t see any problems with any thread doing this.
What suggestions will you have?
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
We block the metastore thread for as long as necessary, we wait for several operations that must be completed in the current revision (creating index storage and obtaining a primary replica). After that, we simply add a task to build the index, this is not a long and complicated operation, I don’t see any problems with any thread doing this.
What suggestions will you have?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348382849
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -853,17 +872,23 @@ private RuntimeException handleStartException(Throwable e) {
LOG.debug(errMsg, e);
- lifecycleManager.stopNode();
+ try {
+ stop();
+ } catch (Exception exception) {
+ exception.addSuppressed(exception);
+ }
return new IgniteException(errMsg, e);
}
- /**
- * Stops ignite node.
- */
- public void stop() {
- lifecycleManager.stopNode();
- restAddressReporter.removeReport();
+ /** Stops ignite node. */
+ public void stop() throws Exception {
+ IgniteUtils.closeAll(
+ lifecycleManager::stopNode,
+ restAddressReporter::removeReport,
+ indexBuilder::close,
+ indexBuildController::close
Review Comment:
Maybe your classes should become ignite components in this case? I don't believe that you don't see how you break existing approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348381751
##########
modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItDataSchemaSyncTest.java:
##########
@@ -62,14 +60,9 @@
*/
@ExtendWith(WorkDirectoryExtension.class)
public class ItDataSchemaSyncTest extends IgniteAbstractTest {
- /**
- * Table name.
- */
public static final String TABLE_NAME = "tbl1";
- /**
- * Nodes bootstrap configuration.
- */
+ /** Nodes bootstrap configuration. */
Review Comment:
This is not a good answer. I asked "why", please reply. I don't understand this inconsistent approach to changes. Why this comment was important, and other weren't? What's the difference? Why did you touch them in the first place?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348405432
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
If you are talking about `completeExceptionally`, I tried to do this, then a compilation error happens, because I need to return `CompletableFuture<ReplicaMeta>` and it says that `CompletableFuture<Object>` will be returned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348458788
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
Are you being serious right now? You're saying that `return CompletableFuture.failedFuture(unwrapThrowable);` lead to compilation error?
> If you are talking about completeExceptionally
No, I'm not talking about that method, why would you assume that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348449218
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
Review Comment:
> and while the storage was being received, the replica expired
It's not possible, right? You said these operations are sequential
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348625271
##########
modules/index/src/test/java/org/apache/ignite/internal/index/IndexBuildControllerTest.java:
##########
@@ -0,0 +1,280 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
+import static org.apache.ignite.internal.table.TableTestUtils.createHashIndex;
+import static org.apache.ignite.internal.table.TableTestUtils.getIndexIdStrict;
+import static org.apache.ignite.internal.table.TableTestUtils.getTableIdStrict;
+import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
+import static org.apache.ignite.sql.ColumnType.INT32;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogTestUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.event.AbstractEventProducer;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.hlc.HybridClockImpl;
+import org.apache.ignite.internal.hlc.HybridTimestamp;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.ReplicationGroupId;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.TableTestUtils;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.testframework.BaseIgniteAbstractTest;
+import org.apache.ignite.internal.util.IgniteUtils;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+import org.apache.ignite.network.TopologyService;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+/** For {@link IndexBuildController} testing. */
+public class IndexBuildControllerTest extends BaseIgniteAbstractTest {
+ private static final String NODE_NAME = "test_node";
+
+ private static final String TABLE_NAME = "test_table";
+
+ private static final String COLUMN_NAME = "test_column";
+
+ private static final String INDEX_NAME = "test_index";
+
+ private ClusterNode localNode;
+
+ private IndexBuilder indexBuilder;
+
+ private IndexManager indexManager;
+
+ private CatalogManager catalogManager;
+
+ private ClusterService clusterService;
+
+ private final TestPlacementDriver placementDriver = new TestPlacementDriver();
+
+ private final HybridClock clock = new HybridClockImpl();
+
+ @BeforeEach
+ void setUp() {
+ localNode = mock(ClusterNode.class, invocation -> NODE_NAME);
+
+ indexBuilder = mock(IndexBuilder.class);
+
+ indexManager = mock(IndexManager.class, invocation -> {
+ MvTableStorage mvTableStorage = mock(MvTableStorage.class);
+ MvPartitionStorage mvPartitionStorage = mock(MvPartitionStorage.class);
+ IndexStorage indexStorage = mock(IndexStorage.class);
+
+ when(mvTableStorage.getMvPartition(anyInt())).thenReturn(mvPartitionStorage);
+ when(mvTableStorage.getIndex(anyInt(), anyInt())).thenReturn(indexStorage);
+
+ return completedFuture(mvTableStorage);
+ });
+
+ clusterService = mock(ClusterService.class, invocation -> mock(TopologyService.class, invocation1 -> localNode));
+
+ catalogManager = CatalogTestUtils.createTestCatalogManager(NODE_NAME, clock);
+ catalogManager.start();
+
+ TableTestUtils.createTable(
+ catalogManager,
+ DEFAULT_SCHEMA_NAME,
+ DEFAULT_ZONE_NAME,
+ TABLE_NAME,
+ List.of(ColumnParams.builder().name(COLUMN_NAME).type(INT32).build()),
+ List.of(COLUMN_NAME)
+ );
+ }
+
+ @AfterEach
+ void tearDown() throws Exception {
+ IgniteUtils.stopAll(catalogManager);
+ }
+
+ @Test
+ void testStartBuildIndexesOnIndexCreate() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ clearInvocations(indexBuilder);
+
+ createIndex(INDEX_NAME);
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStartBuildIndexesOnPrimaryReplicaElected() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(INDEX_NAME)), any(), any(), eq(localNode));
+ verify(indexBuilder).startBuildIndex(eq(tableId()), eq(10), eq(indexId(TABLE_NAME + "_PK")), any(), any(), eq(localNode));
+ }
+
+ @Test
+ void testStopBuildIndexesOnIndexDrop() {
+ newIndexBuildController();
+
+ createIndex(INDEX_NAME);
+
+ int indexId = indexId(INDEX_NAME);
+
+ dropIndex(INDEX_NAME);
+
+ verify(indexBuilder).stopBuildingIndexes(indexId);
+ }
+
+ @Test
+ void testStopBuildIndexesOnChangePrimaryReplica() {
+ newIndexBuildController();
+
+ setPrimaryReplicaForOneSecond(10, NODE_NAME, clock.now());
+ setPrimaryReplicaForOneSecond(10, NODE_NAME + "_other", clock.now());
+
+ verify(indexBuilder).stopBuildingIndexes(tableId(), 10);
+ }
+
+ private void createIndex(String indexName) {
+ createHashIndex(catalogManager, DEFAULT_SCHEMA_NAME, TABLE_NAME, indexName, List.of(COLUMN_NAME), false);
+ }
+
+ private void dropIndex(String indexName) {
+ TableTestUtils.dropIndex(catalogManager, DEFAULT_SCHEMA_NAME, indexName);
+ }
+
+ private void setPrimaryReplicaForOneSecond(int partitionId, String leaseholder, HybridTimestamp startTime) {
Review Comment:
try fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348804414
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
+ primaryReplicaId,
+ parameters.indexDescriptor(),
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+
+ startBuildIndexFutures.add(startBuildIndexFuture);
+ }
+ }
+
+ return allOf(startBuildIndexFutures.toArray(CompletableFuture[]::new));
+ });
+ }
+
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ indexBuilder.stopBuildingIndexes(parameters.indexId());
+
+ return completedFuture(null);
+ });
+ }
+
+ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ TablePartitionId primaryReplicaId = (TablePartitionId) parameters.groupId();
+
+ if (isLocalNode(parameters.leaseholder())) {
+ primaryReplicaIds.add(primaryReplicaId);
+
+ // It is safe to get the latest version of the catalog because the PRIMARY_REPLICA_ELECTED event is handled on the
+ // metastore thread.
+ int catalogVersion = catalogService.latestCatalogVersion();
+
+ return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ catalogVersion,
+ primaryReplicaId,
+ mvTableStorage,
+ replicaMeta
+ ))
+ );
+ } else {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return completedFuture(null);
+ }
+ });
+ }
+
+ private void tryStartBuildIndexesForNewPrimaryReplica(
+ int catalogVersion,
+ TablePartitionId primaryReplicaId,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ // TODO: IGNITE-20530 We only need to get write-only indexes
+ for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
+ if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ }
+ }
+ });
+ }
+
+ private void tryStartBuildIndex(
+ TablePartitionId primaryReplicaId,
+ CatalogIndexDescriptor indexDescriptor,
+ MvTableStorage mvTableStorage,
+ ReplicaMeta replicaMeta
+ ) {
+ inBusyLock(busyLock, () -> {
+ if (isLeaseExpire(replicaMeta)) {
+ stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+
+ return;
+ }
+
+ startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ });
+ }
+
+ private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ if (primaryReplicaIds.remove(replicaId)) {
+ // Primary replica is no longer current, we need to stop building indexes for it.
+ indexBuilder.stopBuildingIndexes(replicaId.tableId(), replicaId.partitionId());
+ }
+ }
+
+ private CompletableFuture<MvTableStorage> getMvTableStorageFuture(long causalityToken, TablePartitionId replicaId) {
+ return indexManager.getMvTableStorage(causalityToken, replicaId.tableId());
+ }
+
+ private CompletableFuture<ReplicaMeta> awaitPrimaryReplicaFowNow(TablePartitionId replicaId) {
+ return placementDriver
+ .awaitPrimaryReplica(replicaId, clock.now(), AWAIT_PRIMARY_REPLICA_TIMEOUT, SECONDS)
+ .handle((replicaMeta, throwable) -> {
+ if (throwable != null) {
+ Throwable unwrapThrowable = ExceptionUtils.unwrapCause(throwable);
+
+ if (unwrapThrowable instanceof PrimaryReplicaAwaitTimeoutException) {
+ return awaitPrimaryReplicaFowNow(replicaId);
+ } else {
+ throw new CompletionException(unwrapThrowable);
Review Comment:
fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347359089
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
I don’t quite understand what you are proposing, should I describe the exception in the documentation? create a new exception? swallow exception from `node.stop()`?
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgnitionImpl.java:
##########
@@ -103,13 +104,16 @@ public CompletableFuture<Ignite> start(
);
}
- /** {@inheritDoc} */
@Override
public void stop(String nodeName) {
readyForInitNodes.remove(nodeName);
nodes.computeIfPresent(nodeName, (name, node) -> {
- node.stop();
+ try {
+ node.stop();
+ } catch (Exception e) {
+ throw new IgniteException(Common.NODE_STOPPING_ERR, e);
Review Comment:
I don’t quite understand what you are proposing, should I describe the exception in the documentation? create a new exception? swallow exception from `node.stop()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347355332
##########
modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java:
##########
@@ -578,6 +586,17 @@ public class IgniteImpl implements Ignite {
indexManager = new IndexManager(schemaManager, distributedTblMgr, catalogManager, metaStorageMgr, registry);
+ indexBuilder = new IndexBuilder(name, Runtime.getRuntime().availableProcessors(), replicaSvc);
+
+ indexBuildController = new IndexBuildController(
+ indexBuilder,
Review Comment:
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "tkalkirill (via GitHub)" <gi...@apache.org>.
tkalkirill commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1347353195
##########
modules/placement-driver/src/main/java/org/apache/ignite/internal/placementdriver/PlacementDriverManager.java:
##########
@@ -258,16 +245,6 @@ private void recoverInternalComponentsBusy() {
long recoveryRevision = recoveryFinishedFuture.join();
- CompletableFuture<Void> startLeaserTrackerFuture = leaseTracker.startTrackAsync(recoveryRevision);
-
- // Forces to wait until recovery is complete before the metastore watches is deployed to avoid races with other components.
- startVv.update(recoveryRevision, (unused, throwable) -> startLeaserTrackerFuture)
- .whenComplete((unused, throwable) -> {
- if (throwable != null) {
- LOG.error("Error starting the PlacementDriverManager internal components", throwable);
- } else {
- LOG.debug("Internal components of the PlacementDriverManager started successfully");
- }
- });
+ leaseTracker.startTrackAsync(recoveryRevision);
Review Comment:
We don't need to fire recovery during recovery, so we don't need to bring back the future anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1348347278
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.index;
+
+import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
+import static org.apache.ignite.internal.util.IgniteUtils.inBusyLockAsync;
+
+import java.util.ArrayList;
+import java.util.Set;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.CompletionException;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Function;
+import org.apache.ignite.internal.catalog.CatalogService;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
+import org.apache.ignite.internal.close.ManuallyCloseable;
+import org.apache.ignite.internal.hlc.HybridClock;
+import org.apache.ignite.internal.placementdriver.PlacementDriver;
+import org.apache.ignite.internal.placementdriver.PrimaryReplicaAwaitTimeoutException;
+import org.apache.ignite.internal.placementdriver.ReplicaMeta;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEvent;
+import org.apache.ignite.internal.placementdriver.event.PrimaryReplicaEventParameters;
+import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.internal.storage.engine.MvTableStorage;
+import org.apache.ignite.internal.storage.index.IndexStorage;
+import org.apache.ignite.internal.table.distributed.index.IndexBuilder;
+import org.apache.ignite.internal.util.ExceptionUtils;
+import org.apache.ignite.internal.util.IgniteSpinBusyLock;
+import org.apache.ignite.network.ClusterNode;
+import org.apache.ignite.network.ClusterService;
+
+/**
+ * Сomponent is responsible for starting and stopping the building of indexes on primary replicas.
+ *
+ * <p>Component handles the following events (indexes are started and stopped by {@link CatalogIndexDescriptor#tableId()} ==
+ * {@link TablePartitionId#tableId()}): </p>
+ * <ul>
+ * <li>{@link CatalogEvent#INDEX_CREATE} - starts building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link CatalogEvent#INDEX_DROP} - stops building indexes for the corresponding local primary replicas.</li>
+ * <li>{@link PrimaryReplicaEvent#PRIMARY_REPLICA_ELECTED} - for a new local primary replica, starts the building of all corresponding
+ * indexes, for an expired primary replica, stops the building of all corresponding indexes.</li>
+ * </ul>
+ */
+// TODO: IGNITE-20544 Start building indexes on node recovery
+public class IndexBuildController implements ManuallyCloseable {
+ private static final long AWAIT_PRIMARY_REPLICA_TIMEOUT = 10;
+
+ private final IndexBuilder indexBuilder;
+
+ private final IndexManager indexManager;
+
+ private final CatalogService catalogService;
+
+ private final ClusterService clusterService;
+
+ private final PlacementDriver placementDriver;
+
+ private final HybridClock clock;
+
+ private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
+
+ private final AtomicBoolean closeGuard = new AtomicBoolean();
+
+ private final Set<TablePartitionId> primaryReplicaIds = ConcurrentHashMap.newKeySet();
+
+ /** Constructor. */
+ public IndexBuildController(
+ IndexBuilder indexBuilder,
+ IndexManager indexManager,
+ CatalogService catalogService,
+ ClusterService clusterService,
+ PlacementDriver placementDriver,
+ HybridClock clock
+ ) {
+ this.indexBuilder = indexBuilder;
+ this.indexManager = indexManager;
+ this.catalogService = catalogService;
+ this.clusterService = clusterService;
+ this.placementDriver = placementDriver;
+ this.clock = clock;
+
+ addListeners();
+ }
+
+ @Override
+ public void close() {
+ if (!closeGuard.compareAndSet(false, true)) {
+ return;
+ }
+
+ busyLock.block();
+ }
+
+ private void addListeners() {
+ catalogService.listen(CatalogEvent.INDEX_CREATE, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexCreate(((CreateIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ catalogService.listen(CatalogEvent.INDEX_DROP, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onIndexDrop(((DropIndexEventParameters) parameters)).thenApply(unused -> false);
+ });
+
+ placementDriver.listen(PrimaryReplicaEvent.PRIMARY_REPLICA_ELECTED, (parameters, exception) -> {
+ if (exception != null) {
+ return failedFuture(exception);
+ }
+
+ return onPrimaryReplicaElected(parameters).thenApply(unused -> false);
+ });
+ }
+
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters parameters) {
+ return inBusyLockAsync(busyLock, () -> {
+ var startBuildIndexFutures = new ArrayList<CompletableFuture<?>>();
+
+ for (TablePartitionId primaryReplicaId : primaryReplicaIds) {
+ if (primaryReplicaId.tableId() == parameters.indexDescriptor().tableId()) {
+ CompletableFuture<?> startBuildIndexFuture = getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryStartBuildIndex(
Review Comment:
tryStartBuildIndex, for example. It's hard to tell what it does and what thread is used. The goal is to block metastorage thread as little as possible, first of all. Second, you're executing the action in the thread that completes "awaitPrimaryReplicaFowNow". What thread is it? Are you allowed to execute your operations there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] IGNITE-20330 Create an abstraction for building indexes [ignite-3]
Posted by "ibessonov (via GitHub)" <gi...@apache.org>.
ibessonov commented on code in PR #2631:
URL: https://github.com/apache/ignite-3/pull/2631#discussion_r1350087068
##########
modules/index/src/main/java/org/apache/ignite/internal/index/IndexBuildController.java:
##########
@@ -185,62 +191,70 @@ private CompletableFuture<?> onPrimaryReplicaElected(PrimaryReplicaEventParamete
int catalogVersion = catalogService.latestCatalogVersion();
return getMvTableStorageFuture(parameters.causalityToken(), primaryReplicaId)
- .thenCompose(mvTableStorage -> awaitPrimaryReplicaFowNow(primaryReplicaId)
- .thenAccept(replicaMeta -> tryStartBuildIndexesForNewPrimaryReplica(
+ .thenCompose(mvTableStorage -> awaitPrimaryReplicaForNow(primaryReplicaId)
+ .thenAccept(replicaMeta -> tryScheduleBuildIndexesForNewPrimaryReplica(
catalogVersion,
primaryReplicaId,
mvTableStorage,
replicaMeta
))
);
} else {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return completedFuture(null);
}
});
}
- private void tryStartBuildIndexesForNewPrimaryReplica(
+ private void tryScheduleBuildIndexesForNewPrimaryReplica(
int catalogVersion,
TablePartitionId primaryReplicaId,
MvTableStorage mvTableStorage,
ReplicaMeta replicaMeta
) {
inBusyLock(busyLock, () -> {
if (isLeaseExpire(replicaMeta)) {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return;
}
// TODO: IGNITE-20530 We only need to get write-only indexes
for (CatalogIndexDescriptor indexDescriptor : catalogService.indexes(catalogVersion)) {
if (primaryReplicaId.tableId() == indexDescriptor.tableId()) {
- startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ scheduleBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
}
}
});
}
- private void tryStartBuildIndex(
+ private void tryScheduleBuildIndex(
TablePartitionId primaryReplicaId,
CatalogIndexDescriptor indexDescriptor,
MvTableStorage mvTableStorage,
ReplicaMeta replicaMeta
) {
inBusyLock(busyLock, () -> {
if (isLeaseExpire(replicaMeta)) {
- stopBuildingIndexesIfPrimacyLost(primaryReplicaId);
+ stopBuildingIndexesIfIfPrimaryExpired(primaryReplicaId);
return;
}
- startBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
+ scheduleBuildIndex(primaryReplicaId, indexDescriptor, mvTableStorage);
});
}
- private void stopBuildingIndexesIfPrimacyLost(TablePartitionId replicaId) {
+ /**
+ * Stops building indexes for a replica that has expired, if it has not been done before.
+ *
+ * <p>We need to stop building indexes at the event of a change of primary replica or after executing asynchronous code, we understand
+ * that the {@link ReplicaMeta#getExpirationTime() expiration time} has come.</p>
+ *
+ * @param replicaId Replica ID.
+ */
+ private void stopBuildingIndexesIfIfPrimaryExpired(TablePartitionId replicaId) {
Review Comment:
Double-`If`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org