You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by am...@apache.org on 2023/07/27 16:50:16 UTC
[ignite-3] branch ignite-19499 updated (097263b8a6 -> a4f0762921)
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a change to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
omit 097263b8a6 wip. Replace direct proxy with waiting for latest catalog version.
omit 9d6dd5ea92 Calculate index storage descriptors synchronously on table start.
omit 7a7174a5c7 Get rid of ZonesConfig from TableManager.
omit 1a0854747c Split catalogVersion into schemaVersion and catalogVersion
omit f01f0138d4 Fix column index mapping.
omit 065c9a102a wip. Use catalog version for cache plans.
omit cbfe31714e wip. Minors after review.
omit e6a0496fca wip. switch to CalciteSqlSchemaManager. Drop dead code.
omit 5578c092df Fix dependency resolving in SQL.
omit 81c8ef0b97 Switch other managers to Catalog events.
omit cf9f535e59 Switch TableManager to Catalog events.
omit f151119f2b Resolve conflicts.
omit a3c253a40f Minor.
omit 38b512c5bd Get rid of DdlCommandHandlerWrapper. Get rid of unused methods. Fix styles.
omit a21adc78b2 Mirror Index changes from Config to Catalog
omit fe28c65cce Mirror Table changes from Config to Catalog.
omit 8392fe8631 Mirror DistributionZone changes from Config to Catalog.
add 6cc42b6a2b IGNITE-20036 Fix resource leaks in client tests and streamer (#2350)
add 5d44dc1617 IGNITE-19397 Return error to client when outdated schema is used (#2317)
add ee843bf9c8 IGNITE-19952 Bump Mockito version to support Java 17 (#2306)
add 55f33041f8 IGNITE-19926 Close CompletedFileUploads when unit is deployed (#2334)
add 8d96ed95ff IGNITE-20040 Fix flaky "testOneChannelLeftIfConnectToEachOther" (#2352)
add c67ad9d99a IGNITE-20039 Add functionality to switch the IndexManager to catalog events #2 (#2354)
add c53feb997a IGNITE-20026 Java client: Remove port range limitation (#2353)
add a6f1f8208b IGNITE-20045 Use classes from Catalog domain in assignments recalculation code (#2358)
add fb100d1027 IGNITE-20043 Prevent certain Mockito-related memory leaks in tests. (#2355)
add a1e85f9ae3 IGNITE-20062 Temporary fix for StackOverflowError in Replica#waitForActualState when a peer leaves (#2363)
add 9d7383ab6d IGNITE-20064 Fix hang in TopologyAwareRaftGroupServiceTest (#2366)
add 8742839b1e IGNITE-20071 Simplify SchemaManager and SchemaRegistryImpl interaction (#2369)
new 5cf3c4161e Mirror DistributionZone changes from Config to Catalog.
new 95758a1f8f Mirror Table changes from Config to Catalog.
new d4573598cc Mirror Index changes from Config to Catalog
new 87585a6643 Get rid of DdlCommandHandlerWrapper. Get rid of unused methods. Fix styles.
new 113bc82748 Minor.
new 9a9c51b61a Resolve conflicts.
new 8221157b78 Switch TableManager to Catalog events.
new e1f9fa9079 Switch other managers to Catalog events.
new e1d5c80761 Fix dependency resolving in SQL.
new 1d4bb28092 wip. switch to CalciteSqlSchemaManager. Drop dead code.
new df1e714f76 wip. Minors after review.
new 06d31d44af wip. Use catalog version for cache plans.
new 0f46f6fdde Fix column index mapping.
new 2a414351cb Split catalogVersion into schemaVersion and catalogVersion
new 09ae21f363 Get rid of ZonesConfig from TableManager.
new 9a55705ccf Calculate index storage descriptors synchronously on table start.
new 605b8f6b29 wip. Replace direct proxy with waiting for latest catalog version.
new 8680b28248 wip.
new a4f0762921 Resolve conflicts
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (097263b8a6)
\
N -- N -- N refs/heads/ignite-19499 (a4f0762921)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 19 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
buildscripts/java-integration-test.gradle | 1 -
gradle/libs.versions.toml | 4 +-
.../apache/ignite/internal/catalog/Catalog.java | 14 +-
...logServiceImpl.java => CatalogManagerImpl.java} | 41 +-
.../ignite/internal/catalog/CatalogService.java | 26 +-
.../descriptors/CatalogSchemaDescriptor.java | 5 +-
.../internal/catalog/storage/UpdateLogImpl.java | 19 +-
...ceSelfTest.java => CatalogManagerSelfTest.java} | 628 +++++++++++----------
.../catalog/storage/UpdateLogImplTest.java | 94 +--
.../internal/client/proto/ClientMessagePacker.java | 3 +-
...ClientMsgPackType.java => ErrorExtensions.java} | 10 +-
.../client/handler/ItClientHandlerMetricsTest.java | 2 +-
.../handler/ClientInboundMessageHandler.java | 23 +-
.../handler/JdbcQueryEventHandlerImplTest.java | 4 +-
modules/client/build.gradle | 2 +
.../ignite/internal/client/TcpClientChannel.java | 5 +-
.../apache/ignite/client/AbstractClientTest.java | 4 +-
.../ignite/client/ClientAuthenticationTest.java | 6 +-
.../apache/ignite/client/ClientMetricsTest.java | 35 +-
.../org/apache/ignite/client/DataStreamerTest.java | 83 ++-
.../ignite/client/PartitionAwarenessTest.java | 122 ++--
.../ignite/client/fakes/FakeSchemaRegistry.java | 5 +
.../internal/deployunit/DefaultNodeCallback.java | 16 +-
.../internal/deployunit/DeploymentManagerImpl.java | 41 +-
.../ignite/internal/deployunit/DeploymentUnit.java | 19 +-
.../internal/deployunit/FileDeployerService.java | 12 +-
.../internal/deployunit/IgniteDeployment.java | 4 +-
.../ignite/internal/deployunit/UnitContent.java | 24 +-
.../ignite/deployment/FileDeployerServiceTest.java | 41 +-
.../compute/util/DummyIgniteDeployment.java | 2 +-
modules/core/build.gradle | 1 -
.../internal/streamer/StreamerSubscriber.java | 34 +-
.../java/org/apache/ignite/lang/ErrorGroups.java | 3 +
.../testframework/BaseIgniteAbstractTest.java | 10 +
modules/distribution-zones/build.gradle | 1 +
.../rebalance/DistributionZoneRebalanceEngine.java | 26 +-
.../distributionzones/rebalance/RebalanceUtil.java | 42 +-
.../rebalance/ZoneCatalogDescriptorUtils.java | 68 +++
.../DistributionZoneRebalanceEngineTest.java | 25 +
.../RebalanceUtilUpdateAssignmentsTest.java | 21 +-
modules/index/build.gradle | 1 -
.../metastorage/server/AndConditionTest.java | 2 +-
.../network/netty/ItConnectionManagerTest.java | 9 +-
.../dotnet/Apache.Ignite.Tests/FakeServer.cs | 1 +
.../dotnet/Apache.Ignite/Internal/ClientSocket.cs | 3 +
.../ItPlacementDriverReplicaSideTest.java | 1 +
.../apache/ignite/internal/replicator/Replica.java | 23 +-
.../ignite/internal/replicator/ReplicaManager.java | 31 +-
.../client/TopologyAwareRaftGroupServiceTest.java | 12 +-
.../replicator/PlacementDriverReplicaSideTest.java | 24 +-
.../deployment/CompletedFileUploadSubscriber.java | 35 +-
.../deployment/DeploymentManagementController.java | 23 +-
.../internal/compute/ItComputeTestStandalone.java | 2 +-
.../storage/ItRebalanceDistributedTest.java | 5 +-
.../ignite/internal/deployment/DeployFiles.java | 14 +-
.../deployment/ItDeploymentUnitFailoverTest.java | 2 +
.../runner/app/ItIgniteNodeRestartTest.java | 5 +-
.../app/client/ItAbstractThinClientTest.java | 19 +
.../ItThinClientSchemaSynchronizationTest.java | 58 ++
.../runner/app/client/ItThinClientSqlTest.java | 20 -
.../streamer/ItAbstractDataStreamerTest.java | 88 +--
.../streamer/ItClientDataStreamerTest.java | 6 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 5 +-
.../ignite/internal/schema/SchemaException.java | 10 +
.../ignite/internal/schema/SchemaManager.java | 145 ++---
.../ignite/internal/schema/SchemaRegistry.java | 9 +-
.../schema/SchemaVersionMismatchException.java | 65 +++
.../schema/registry/SchemaRegistryImpl.java | 31 +-
.../schema/registry/SchemaRegistryImplTest.java | 46 +-
modules/sql-engine/build.gradle | 1 -
.../internal/sql/engine/SqlQueryProcessor.java | 5 -
.../sql/engine/exec/ExecutionServiceImpl.java | 3 +-
.../sql/engine/exec/ddl/DdlCommandHandler.java | 34 +-
.../engine/rule/logical/LogicalOrToUnionRule.java | 4 +-
.../sql/engine/trait/DistributionFunction.java | 4 +-
.../sql/engine/util/HashFunctionFactoryImpl.java | 27 +-
.../DdlCommandHandlerExceptionHandlingTest.java | 8 +-
.../ddl/DistributionZoneDdlCommandHandlerTest.java | 8 +-
modules/table/build.gradle | 1 -
.../distributed/ItTxDistributedTestSingleNode.java | 1 +
.../ignite/distributed/ReplicaUnavailableTest.java | 7 +-
.../schema/marshaller/TupleMarshallerImpl.java | 14 +-
.../internal/table/distributed/TableManager.java | 19 -
.../TupleMarshallerFixlenOnlyBenchmark.java | 6 +-
.../TupleMarshallerVarlenOnlyBenchmark.java | 6 +-
.../table/distributed/LowWatermarkTest.java | 3 +-
.../table/impl/DummySchemaManagerImpl.java | 5 +
87 files changed, 1461 insertions(+), 916 deletions(-)
rename modules/catalog/src/main/java/org/apache/ignite/internal/catalog/{CatalogServiceImpl.java => CatalogManagerImpl.java} (96%)
rename modules/catalog/src/test/java/org/apache/ignite/internal/catalog/{CatalogServiceSelfTest.java => CatalogManagerSelfTest.java} (77%)
copy modules/client-common/src/main/java/org/apache/ignite/internal/client/proto/{ClientMsgPackType.java => ErrorExtensions.java} (77%)
create mode 100644 modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/rebalance/ZoneCatalogDescriptorUtils.java
create mode 100644 modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/client/ItThinClientSchemaSynchronizationTest.java
create mode 100644 modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaVersionMismatchException.java
[ignite-3] 14/19: Split catalogVersion into schemaVersion and catalogVersion
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 2a414351cbe9177249204ed1d3d86503ca83cf65
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 21 15:48:51 2023 +0300
Split catalogVersion into schemaVersion and catalogVersion
---
.../apache/ignite/internal/catalog/commands/CatalogUtils.java | 1 +
.../internal/catalog/descriptors/CatalogTableDescriptor.java | 8 ++++++++
.../ignite/internal/catalog/storage/AlterColumnEntry.java | 1 +
.../ignite/internal/catalog/storage/DropColumnsEntry.java | 1 +
.../ignite/internal/catalog/storage/NewColumnsEntry.java | 1 +
.../apache/ignite/internal/schema/CatalogDescriptorUtils.java | 5 +++--
.../java/org/apache/ignite/internal/schema/SchemaManager.java | 7 ++-----
.../java/org/apache/ignite/internal/schema/SchemaUtils.java | 11 -----------
.../internal/sql/engine/exec/ExecutableTableRegistryImpl.java | 5 ++++-
.../ignite/internal/sql/engine/exec/MockedStructuresTest.java | 5 -----
.../sql/engine/schema/CatalogSqlSchemaManagerTest.java | 2 +-
.../ignite/internal/table/distributed/TableManager.java | 2 +-
.../ignite/internal/table/distributed/TableManagerTest.java | 5 -----
13 files changed, 23 insertions(+), 31 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index 4243a54b3d..fa918d9e83 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -87,6 +87,7 @@ public class CatalogUtils {
return new CatalogTableDescriptor(
id,
params.tableName(),
+ 1,
zoneId,
params.columns().stream().map(CatalogUtils::fromParams).collect(toList()),
params.primaryKeyColumns(),
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogTableDescriptor.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogTableDescriptor.java
index 58a744d74c..e438721f18 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogTableDescriptor.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogTableDescriptor.java
@@ -35,6 +35,7 @@ import org.jetbrains.annotations.Nullable;
public class CatalogTableDescriptor extends CatalogObjectDescriptor {
private static final long serialVersionUID = -2021394971104316570L;
+ private final int version;
private final int zoneId;
private final List<CatalogTableColumnDescriptor> columns;
@@ -49,6 +50,7 @@ public class CatalogTableDescriptor extends CatalogObjectDescriptor {
*
* @param id Table id.
* @param name Table name.
+ * @param version Table descriptor version.
* @param zoneId Distribution zone ID.
* @param columns Table column descriptors.
* @param pkCols Primary key column names.
@@ -57,6 +59,7 @@ public class CatalogTableDescriptor extends CatalogObjectDescriptor {
public CatalogTableDescriptor(
int id,
String name,
+ int version,
int zoneId,
List<CatalogTableColumnDescriptor> columns,
List<String> pkCols,
@@ -64,6 +67,7 @@ public class CatalogTableDescriptor extends CatalogObjectDescriptor {
) {
super(id, Type.TABLE, name);
+ this.version = version;
this.zoneId = zoneId;
this.columns = Objects.requireNonNull(columns, "No columns defined.");
primaryKeyColumns = Objects.requireNonNull(pkCols, "No primary key columns.");
@@ -85,6 +89,10 @@ public class CatalogTableDescriptor extends CatalogObjectDescriptor {
return columnsMap.get(columnName);
}
+ public int version() {
+ return version;
+ }
+
public int zoneId() {
return zoneId;
}
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/AlterColumnEntry.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/AlterColumnEntry.java
index 910a363d81..b2e7a11ada 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/AlterColumnEntry.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/AlterColumnEntry.java
@@ -91,6 +91,7 @@ public class AlterColumnEntry implements UpdateEntry, Fireable {
: new CatalogTableDescriptor(
table.id(),
table.name(),
+ table.version() + 1,
table.zoneId(),
table.columns().stream()
.map(source -> source.name().equals(column.name()) ? column : source)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/DropColumnsEntry.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/DropColumnsEntry.java
index e29c250283..562db4122a 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/DropColumnsEntry.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/DropColumnsEntry.java
@@ -88,6 +88,7 @@ public class DropColumnsEntry implements UpdateEntry, Fireable {
.map(table -> table.id() == tableId ? new CatalogTableDescriptor(
table.id(),
table.name(),
+ table.version() + 1,
table.zoneId(),
table.columns().stream()
.filter(col -> !columns.contains(col.name()))
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/NewColumnsEntry.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/NewColumnsEntry.java
index 1e1aff017e..ca93e4a358 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/NewColumnsEntry.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/NewColumnsEntry.java
@@ -88,6 +88,7 @@ public class NewColumnsEntry implements UpdateEntry, Fireable {
.map(table -> table.id() == tableId ? new CatalogTableDescriptor(
table.id(),
table.name(),
+ table.version() + 1,
table.zoneId(),
CollectionUtils.concat(table.columns(), descriptors),
table.primaryKeyColumns(),
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
index e400dd1d11..5959b0ddf6 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
@@ -74,6 +74,7 @@ public class CatalogDescriptorUtils {
return new CatalogTableDescriptor(
config.id(),
config.name(),
+ 1,
config.zoneId(),
config.columns().stream().map(CatalogDescriptorUtils::toTableColumnDescriptor).collect(toList()),
List.of(primaryKeyConfig.columns()),
@@ -314,7 +315,7 @@ public class CatalogDescriptorUtils {
/**
* Converts table descriptor to table schema descriptor.
*/
- public static SchemaDescriptor convert(int schemaVer, CatalogTableDescriptor tableDescriptor) {
+ public static SchemaDescriptor convert(CatalogTableDescriptor tableDescriptor) {
List<Column> keyCols = new ArrayList<>(tableDescriptor.primaryKeyColumns().size());
List<Column> valCols = new ArrayList<>(tableDescriptor.columns().size() - tableDescriptor.primaryKeyColumns().size());
@@ -331,7 +332,7 @@ public class CatalogDescriptorUtils {
}
return new SchemaDescriptor(
- schemaVer,
+ tableDescriptor.version(),
keyCols.toArray(Column[]::new),
tableDescriptor.colocationColumns().toArray(String[]::new),
valCols.toArray(Column[]::new)
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index d1a2ff5415..af0451b259 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -122,10 +122,9 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
private CompletableFuture<?> onTableCreated(CreateTableEventParameters evt) {
CatalogTableDescriptor tableDescriptor = evt.tableDescriptor();
- int newSchemaVersion = INITIAL_SCHEMA_VERSION; // evt.catalogVersion();
int tblId = tableDescriptor.id();
- SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
+ SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(tableDescriptor);
return onSchemaChange(tblId, newSchema, evt.causalityToken());
}
@@ -160,9 +159,7 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
return completedFuture(new UnsupportedOperationException("Unexpected event."));
}
- int newSchemaVersion = registriesVv.latest().get(tableDescriptor.id()).lastSchemaVersion() + 1;
-
- SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
+ SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(tableDescriptor);
return onSchemaChange(tableDescriptor.id(), newSchema, evt.causalityToken());
} catch (Throwable th) {
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
index aa0bbc51b3..ac4cef9e78 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
@@ -28,17 +28,6 @@ import org.apache.ignite.internal.schema.mapping.ColumnMapping;
* Stateless schema utils that produces helper methods for schema preparation.
*/
public class SchemaUtils {
- /**
- * Creates schema descriptor for the table with specified configuration.
- *
- * @param schemaVer Schema version.
- * @param tableView Table configuration.
- * @return Schema descriptor.
- */
- public static SchemaDescriptor prepareSchemaDescriptor(int schemaVer, TableView tableView) {
- return ConfigurationToSchemaDescriptorConverter.convert(schemaVer, tableView);
- }
-
/**
* Prepares column mapper.
*
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
index 67c9b5c4fd..b84644d8ad 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
@@ -20,6 +20,7 @@ package org.apache.ignite.internal.sql.engine.exec;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentMap;
+import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.replicator.ReplicaService;
import org.apache.ignite.internal.schema.SchemaDescriptor;
@@ -76,10 +77,12 @@ public class ExecutableTableRegistryImpl implements ExecutableTableRegistry, Sch
private CompletableFuture<ExecutableTable> loadTable(int schemaVersion, int tableId, TableDescriptor tableDescriptor) {
return tableManager.tableAsync(tableId)
.thenApply(table -> {
+ CatalogTableDescriptor descriptor = tableManager.catalogManager.table(tableId, schemaVersion);
+
InternalTable internalTable = table.internalTable();
SchemaRegistry schemaRegistry = schemaManager.schemaRegistry(tableId);
- SchemaDescriptor schemaDescriptor = schemaRegistry.schema(schemaVersion);
+ SchemaDescriptor schemaDescriptor = schemaRegistry.schema(descriptor.version());
TableRowConverter rowConverter = new TableRowConverterImpl(schemaRegistry, schemaDescriptor, tableDescriptor);
ScannableTable scannableTable = new ScannableTableImpl(internalTable, rowConverter, tableDescriptor);
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index 26a355f767..6bce878f4f 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -557,11 +557,6 @@ public class MockedStructuresTest extends IgniteAbstractTest {
new NetworkAddress("localhost", 47500)
));
- try (MockedStatic<SchemaUtils> schemaServiceMock = mockStatic(SchemaUtils.class)) {
- schemaServiceMock.when(() -> SchemaUtils.prepareSchemaDescriptor(anyInt(), any()))
- .thenReturn(mock(SchemaDescriptor.class));
- }
-
when(cs.messagingService()).thenAnswer(invocation -> {
MessagingService ret = mock(MessagingService.class);
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
index 894a6a60ba..382adc6162 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
@@ -573,7 +573,7 @@ public class CatalogSqlSchemaManagerTest {
columnDescriptors.add(newCol);
}
- return new CatalogTableDescriptor(id, name, zoneId, columnDescriptors, primaryKey, colocationKey);
+ return new CatalogTableDescriptor(id, name, 1, zoneId, columnDescriptors, primaryKey, colocationKey);
}
}
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index a6dd0dd47e..4d9a5cf7e9 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -328,7 +328,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
private final SchemaManager schemaManager;
/** Catalog manager. */
- private final CatalogManager catalogManager;
+ public final CatalogManager catalogManager;
private final LogStorageFactoryCreator volatileLogStorageFactoryCreator;
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index 7fc676d9b3..a223eddbf1 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -775,11 +775,6 @@ public class TableManagerTest extends IgniteAbstractTest {
new NetworkAddress("localhost", 47500)
));
- try (MockedStatic<SchemaUtils> schemaServiceMock = mockStatic(SchemaUtils.class)) {
- schemaServiceMock.when(() -> SchemaUtils.prepareSchemaDescriptor(anyInt(), any()))
- .thenReturn(mock(SchemaDescriptor.class));
- }
-
try (MockedStatic<AffinityUtils> affinityServiceMock = mockStatic(AffinityUtils.class)) {
ArrayList<List<ClusterNode>> assignment = new ArrayList<>(PARTITIONS);
[ignite-3] 15/19: Get rid of ZonesConfig from TableManager.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 09ae21f3631a3038b63ab24a929dc2058873a1ed
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 21 16:09:18 2023 +0300
Get rid of ZonesConfig from TableManager.
---
.../storage/ItRebalanceDistributedTest.java | 1 -
.../runner/app/ItIgniteNodeRestartTest.java | 1 -
.../org/apache/ignite/internal/app/IgniteImpl.java | 1 -
.../sql/engine/exec/MockedStructuresTest.java | 5 ---
.../internal/table/distributed/TableManager.java | 39 ++++++++--------------
.../table/distributed/TableManagerTest.java | 3 --
6 files changed, 14 insertions(+), 36 deletions(-)
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
index b0bafa6b56..17e82e6d29 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
@@ -801,7 +801,6 @@ public class ItRebalanceDistributedTest {
name,
registry,
tablesCfg,
- zonesCfg,
gcConfig,
clusterService,
raftManager,
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 11aed86bdf..e3cce2caae 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -362,7 +362,6 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
name,
registry,
tablesConfig,
- zonesConfig,
gcConfig,
clusterSvc,
raftMgr,
diff --git a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
index 8c66e13188..11e61b66a6 100644
--- a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
+++ b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
@@ -526,7 +526,6 @@ public class IgniteImpl implements Ignite {
name,
registry,
tablesConfig,
- zonesConfig,
gcConfig,
clusterSvc,
raftMgr,
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index 6bce878f4f..aea8087ad1 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -30,7 +30,6 @@ import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.ArgumentMatchers.any;
-import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
@@ -72,9 +71,7 @@ import org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupServiceFacto
import org.apache.ignite.internal.raft.service.RaftGroupService;
import org.apache.ignite.internal.replicator.ReplicaManager;
import org.apache.ignite.internal.replicator.ReplicaService;
-import org.apache.ignite.internal.schema.SchemaDescriptor;
import org.apache.ignite.internal.schema.SchemaManager;
-import org.apache.ignite.internal.schema.SchemaUtils;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
@@ -117,7 +114,6 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
-import org.mockito.MockedStatic;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
@@ -583,7 +579,6 @@ public class MockedStructuresTest extends IgniteAbstractTest {
"",
revisionUpdater,
tblsCfg,
- dstZnsCfg,
gcConfig,
cs,
rm,
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 4d9a5cf7e9..3d269271fe 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -27,14 +27,11 @@ import static java.util.concurrent.CompletableFuture.supplyAsync;
import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
import static org.apache.ignite.internal.causality.IncrementalVersionedValue.dependingOn;
-import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.getZoneById;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.partitionAssignments;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.tableAssignments;
import static org.apache.ignite.internal.distributionzones.rebalance.ZoneCatalogDescriptorUtils.toZoneDescriptor;
import static org.apache.ignite.internal.metastorage.dsl.Operations.put;
-import static org.apache.ignite.internal.schema.CatalogDescriptorUtils.toTableDescriptor;
import static org.apache.ignite.internal.schema.SchemaManager.INITIAL_SCHEMA_VERSION;
-import static org.apache.ignite.internal.schema.configuration.SchemaConfigurationUtils.findTableView;
import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
import static org.apache.ignite.internal.util.IgniteUtils.shutdownAndAwaitTermination;
@@ -107,7 +104,6 @@ import org.apache.ignite.internal.causality.CompletionListener;
import org.apache.ignite.internal.causality.IncrementalVersionedValue;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
-import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.hlc.HybridTimestamp;
import org.apache.ignite.internal.logger.IgniteLogger;
@@ -144,7 +140,6 @@ import org.apache.ignite.internal.schema.configuration.GcConfiguration;
import org.apache.ignite.internal.schema.configuration.PrimaryKeyView;
import org.apache.ignite.internal.schema.configuration.TableChange;
import org.apache.ignite.internal.schema.configuration.TableConfiguration;
-import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesChange;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
@@ -244,9 +239,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
/** Tables configuration. */
private final TablesConfiguration tablesCfg;
- /** Distribution zones configuration. */
- private final DistributionZonesConfiguration zonesConfig;
-
/** Garbage collector configuration. */
private final GcConfiguration gcConfig;
@@ -396,7 +388,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param nodeName Node name.
* @param registry Registry for versioned values.
* @param tablesCfg Tables configuration.
- * @param zonesConfig Distribution zones configuration.
* @param gcConfig Garbage collector configuration.
* @param raftMgr Raft manager.
* @param replicaMgr Replica manager.
@@ -416,7 +407,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
String nodeName,
Consumer<LongFunction<CompletableFuture<?>>> registry,
TablesConfiguration tablesCfg,
- DistributionZonesConfiguration zonesConfig,
GcConfiguration gcConfig,
ClusterService clusterService,
RaftManager raftMgr,
@@ -440,7 +430,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
DistributionZoneManager distributionZoneManager
) {
this.tablesCfg = tablesCfg;
- this.zonesConfig = zonesConfig;
this.gcConfig = gcConfig;
this.clusterService = clusterService;
this.raftMgr = raftMgr;
@@ -1463,15 +1452,13 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
private Set<Assignment> calculateAssignments(TablePartitionId tablePartitionId) {
- CatalogTableDescriptor tableDescriptor = getTableDescriptor(tablePartitionId.tableId());
-
- assert tableDescriptor != null : tablePartitionId;
+ int replicas = tableReplicas(tablePartitionId.tableId());
return AffinityUtils.calculateAssignmentForPartition(
// TODO: https://issues.apache.org/jira/browse/IGNITE-19425 we must use distribution zone keys here
baselineMgr.nodes().stream().map(ClusterNode::name).collect(toList()),
tablePartitionId.partitionId(),
- getZoneDescriptor(tableDescriptor.zoneId()).replicas()
+ replicas
);
}
@@ -2493,14 +2480,12 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- CatalogTableDescriptor tableDescriptor = getTableDescriptor(tableId);
-
- assert tableDescriptor != null : replicaGrpId;
+ int replicas = tableReplicas(tableId);
return RebalanceUtil.handleReduceChanged(
metaStorageMgr,
baselineMgr.nodes().stream().map(ClusterNode::name).collect(toList()),
- getZoneDescriptor(tableDescriptor.zoneId()).replicas(),
+ replicas,
replicaGrpId,
evt
);
@@ -2771,14 +2756,18 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
return startedTables.get(tableId);
}
- private @Nullable CatalogTableDescriptor getTableDescriptor(int id) {
- TableView tableView = findTableView(tablesCfg.value(), id);
+ private int tableReplicas(int tableId) {
+ int version = catalogManager.activeCatalogVersion(Long.MAX_VALUE);
- return tableView == null ? null : toTableDescriptor(tableView);
- }
+ CatalogTableDescriptor table = catalogManager.table(tableId, version);
+
+ assert table != null;
+
+ CatalogZoneDescriptor zone = catalogManager.zone(table.zoneId(), version);
+
+ assert zone != null;
- private CatalogZoneDescriptor getZoneDescriptor(int id) {
- return toZoneDescriptor(getZoneById(zonesConfig, id).value());
+ return zone.replicas();
}
// Copied from DdlCommandHandler
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index a223eddbf1..c5057c465b 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -92,9 +92,7 @@ import org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupServiceFacto
import org.apache.ignite.internal.raft.service.RaftGroupService;
import org.apache.ignite.internal.raft.storage.impl.LocalLogStorageFactory;
import org.apache.ignite.internal.replicator.ReplicaManager;
-import org.apache.ignite.internal.schema.SchemaDescriptor;
import org.apache.ignite.internal.schema.SchemaManager;
-import org.apache.ignite.internal.schema.SchemaUtils;
import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
import org.apache.ignite.internal.schema.configuration.TableView;
@@ -865,7 +863,6 @@ public class TableManagerTest extends IgniteAbstractTest {
"test",
revisionUpdater,
tblsCfg,
- distributionZonesConfiguration,
gcConfig,
clusterService,
rm,
[ignite-3] 11/19: wip. Minors after review.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit df1e714f76e801601a132b4a786d770617b445df
Author: amashenkov <an...@gmail.com>
AuthorDate: Thu Jul 20 17:42:44 2023 +0300
wip. Minors after review.
---
.../internal/catalog/CatalogManagerImpl.java | 2 +-
.../catalog/commands/CreateTableParams.java | 10 +++---
.../distributionzones/DistributionZoneManager.java | 2 +-
.../apache/ignite/internal/index/IndexManager.java | 8 ++---
.../ignite/internal/index/ItIndexManagerTest.java | 6 ++--
.../ignite/internal/schema/SchemaManager.java | 3 +-
.../internal/table/distributed/TableManager.java | 36 ++++++++++------------
7 files changed, 33 insertions(+), 34 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index d441478fe6..fe30bbcf89 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -311,7 +311,7 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
validateCreateTableParams(params);
- CatalogZoneDescriptor zone = getZone(catalog, Objects.requireNonNullElse(params.zone(), DEFAULT_ZONE_NAME));
+ CatalogZoneDescriptor zone = getZone(catalog, params.zone());
int id = catalog.objectIdGenState();
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CreateTableParams.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CreateTableParams.java
index 525219ae2a..884b8090f4 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CreateTableParams.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CreateTableParams.java
@@ -18,6 +18,8 @@
package org.apache.ignite.internal.catalog.commands;
import java.util.List;
+import java.util.Objects;
+import org.apache.ignite.internal.catalog.CatalogService;
import org.jetbrains.annotations.Nullable;
/**
@@ -41,8 +43,7 @@ public class CreateTableParams extends AbstractTableCommandParams {
private List<ColumnParams> cols;
/** Distribution zone name. */
- @Nullable
- private String zone;
+ private String zone = CatalogService.DEFAULT_ZONE_NAME;
private CreateTableParams() {
}
@@ -72,7 +73,6 @@ public class CreateTableParams extends AbstractTableCommandParams {
/**
* Gets zone name.
*/
- @Nullable
public String zone() {
return zone;
}
@@ -126,8 +126,8 @@ public class CreateTableParams extends AbstractTableCommandParams {
* @param zoneName Zone name.
* @return {@code this}.
*/
- public Builder zone(@Nullable String zoneName) {
- params.zone = zoneName;
+ public Builder zone(String zoneName) {
+ params.zone = Objects.requireNonNullElse(zoneName, CatalogService.DEFAULT_ZONE_NAME);
return this;
}
diff --git a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
index 8f67682499..3e3df85e39 100644
--- a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
+++ b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
@@ -191,7 +191,7 @@ public class DistributionZoneManager implements IgniteComponent {
/** Tables configuration. */
private final TablesConfiguration tablesConfiguration;
- /** Catalog service. */
+ /** Catalog manager. */
private final CatalogManager catalogManager;
/** Meta Storage manager. */
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index 89fee4b074..7e5caf9491 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -18,6 +18,7 @@
package org.apache.ignite.internal.index;
import static java.util.concurrent.CompletableFuture.allOf;
+import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import static org.apache.ignite.internal.util.ArrayUtils.STRING_EMPTY_ARRAY;
@@ -199,10 +200,9 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
try {
- CompletableFuture<?> fireEventFuture = fireEvent(IndexEvent.DROP, new IndexEventParameters(causalityToken, tableId, idxId));
+ fireEvent(IndexEvent.DROP, new IndexEventParameters(causalityToken, tableId, idxId));
- catalogManager.table(tableId, evt.catalogVersion());
- CompletableFuture<?> dropIndexFuture = tableManager.tableAsync(tableId)
+ CompletableFuture<?> dropIndexFuture = tableManager.tableAsync(evt.causalityToken(), tableId)
.thenAccept(table -> {
if (table != null) { // in case of DROP TABLE the table will be removed first
table.unregisterIndex(idxId);
@@ -210,7 +210,7 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
});
// TODO: investigate why DropIndexFuture hangs.
- return allOf(fireEventFuture, dropIndexFuture);
+ return completedFuture(null); // dropIndexFuture;
} catch (Throwable th) {
LOG.warn("Failed to process drop index event.", th);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
index 242c4fbdca..13350c0bd6 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
@@ -27,6 +27,7 @@ import java.util.List;
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.Ignite;
import org.apache.ignite.internal.app.IgniteImpl;
+import org.apache.ignite.internal.catalog.CatalogService;
import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
import org.apache.ignite.internal.catalog.commands.DropIndexParams;
import org.apache.ignite.internal.index.event.IndexEvent;
@@ -72,7 +73,7 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
await(indexManager.createIndexAsync(
CreateHashIndexParams.builder()
- .schemaName("PUBLIC")
+ .schemaName(CatalogService.DEFAULT_SCHEMA_NAME)
.indexName("INAME")
.tableName("TNAME")
.columns(List.of("C3", "C2"))
@@ -93,7 +94,8 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
CompletableFuture<IndexEventParameters> indexDroppedFuture = registerListener(indexManager, IndexEvent.DROP);
- await(indexManager.dropIndexAsync(DropIndexParams.builder().schemaName("PUBLIC").indexName("INAME").build()));
+ await(indexManager.dropIndexAsync(DropIndexParams.builder()
+ .schemaName(CatalogService.DEFAULT_SCHEMA_NAME).indexName("INAME").build()));
{
IndexEventParameters params = await(indexDroppedFuture);
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index 41f13823e2..d1a2ff5415 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -97,7 +97,8 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
public SchemaManager(
Consumer<LongFunction<CompletableFuture<?>>> registry,
MetaStorageManager metastorageMgr,
- CatalogManager catalogManager) {
+ CatalogManager catalogManager
+ ) {
this.registriesVv = new IncrementalVersionedValue<>(registry, HashMap::new);
this.catalogManager = catalogManager;
this.metastorageMgr = metastorageMgr;
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 655c584dff..be426e99f6 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -26,7 +26,6 @@ import static java.util.concurrent.CompletableFuture.runAsync;
import static java.util.concurrent.CompletableFuture.supplyAsync;
import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
-import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
import static org.apache.ignite.internal.causality.IncrementalVersionedValue.dependingOn;
import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.getZoneById;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.partitionAssignments;
@@ -98,6 +97,7 @@ import org.apache.ignite.internal.catalog.commands.ColumnParams;
import org.apache.ignite.internal.catalog.commands.CreateTableParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogObjectDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.catalog.events.CatalogEvent;
@@ -613,10 +613,16 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
assert !assignments.isEmpty() : "Couldn't create the table with empty assignments.";
+ int[] indexesToWait = Arrays.stream(catalogManager.schema(evt.catalogVersion()).indexes())
+ .filter(i -> i.tableId() == tableId)
+ .mapToInt(CatalogObjectDescriptor::id)
+ .toArray();
+
CompletableFuture<?> createTableFut = createTableLocally(
evt.causalityToken(),
tableDescriptor,
zoneDescriptor,
+ indexesToWait,
assignments
).whenComplete((v, e) -> {
if (e == null) {
@@ -1230,12 +1236,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param causalityToken Causality token.
* @param tableDescriptor Catalog table descriptor.
* @param zoneDescriptor Catalog distributed zone descriptor.
+ * @param indexesToWait
* @return Future that will be completed when local changes related to the table creation are applied.
*/
private CompletableFuture<?> createTableLocally(
long causalityToken,
CatalogTableDescriptor tableDescriptor,
CatalogZoneDescriptor zoneDescriptor,
+ int[] indexesToWait,
List<Set<Assignment>> assignments
) {
String tableName = tableDescriptor.name();
@@ -1256,7 +1264,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
var table = new TableImpl(internalTable, lockMgr);
// TODO: IGNITE-19082 Need another way to wait for indexes
- table.addIndexesToWait(collectTableIndexIds(tableId));
+// table.addIndexesToWait(indexesToWait);
tablesByIdVv.update(causalityToken, (previous, e) -> inBusyLock(busyLock, () -> {
if (e != null) {
@@ -1293,7 +1301,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
// TODO: https://issues.apache.org/jira/browse/IGNITE-19913 Possible performance degradation.
fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
- // TODO: investigate why createParts hangs.
+ // TODO: investigate why createParts future hangs.
return completedFuture(false);
}
@@ -1481,7 +1489,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
String tableName = parameters.tableName();
- String zoneName = Objects.requireNonNullElse(parameters.zone(), DEFAULT_ZONE_NAME);
+ String zoneName = parameters.zone();
// Copied from DdlCommandHandler
Consumer<TableChange> tblChanger = tableChange -> {
@@ -1491,19 +1499,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
});
- var colocationKeys = parameters.colocationColumns();
-
- if (nullOrEmpty(colocationKeys)) {
- colocationKeys = parameters.primaryKeyColumns();
- }
-
- var colocationKeys0 = colocationKeys;
+ List<String> colocationKeys = nullOrEmpty(parameters.colocationColumns())
+ ? parameters.primaryKeyColumns()
+ : parameters.colocationColumns();
tableChange.changePrimaryKey(pkChange -> pkChange.changeColumns(parameters.primaryKeyColumns().toArray(String[]::new))
- .changeColocationColumns(colocationKeys0.toArray(String[]::new)));
+ .changeColocationColumns(colocationKeys.toArray(String[]::new)));
};
-
return catalogManager.createTable(parameters)
.thenApply(ignore -> catalogManager.table(tableName, Long.MAX_VALUE).id())
.thenCompose(tableId -> createTableAsyncInternal(tableId, tableName, zoneName, tblChanger));
@@ -2720,13 +2723,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
}
- private int[] collectTableIndexIds(int tableId) {
- return tablesCfg.value().indexes().stream()
- .filter(tableIndexView -> tableIndexView.tableId() == tableId)
- .mapToInt(TableIndexView::id)
- .toArray();
- }
-
private static void closePartitionTrackers(InternalTable internalTable, int partitionId) {
closeTracker(internalTable.getPartitionSafeTimeTracker(partitionId));
[ignite-3] 02/19: Mirror Table changes from Config to Catalog.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 95758a1f8fd0aa3887f524acaa888a26f03f7867
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 7 15:40:08 2023 +0300
Mirror Table changes from Config to Catalog.
---
.../internal/catalog/commands/CatalogUtils.java | 69 ++++++-
.../internal/catalog/commands/ColumnParams.java | 6 +-
.../internal/catalog/CatalogManagerSelfTest.java | 2 +-
.../storage/ItRebalanceDistributedTest.java | 32 ++--
.../inmemory/ItRaftStorageVolatilityTest.java | 12 +-
.../raftsnapshot/ItTableRaftSnapshotsTest.java | 2 +-
.../runner/app/ItIgniteNodeRestartTest.java | 5 +-
.../runner/app/ItSchemaChangeKvViewTest.java | 2 +
.../runner/app/ItSchemaChangeTableViewTest.java | 1 +
.../runner/app/ItTableApiContractTest.java | 124 ++++++------
.../internal/runner/app/ItTablesApiTest.java | 56 +++---
.../runner/app/PlatformTestNodeRunner.java | 18 +-
.../ignite/internal/sql/api/ItCommonApiTest.java | 6 +-
.../internal/sql/api/ItSqlAsynchronousApiTest.java | 22 +--
.../internal/sql/api/ItSqlSynchronousApiTest.java | 24 +--
.../internal/sql/engine/ItCreateTableDdlTest.java | 30 +--
.../ignite/internal/table/ItRoReadsTest.java | 24 ++-
.../org/apache/ignite/internal/app/IgniteImpl.java | 1 +
modules/schema/build.gradle | 1 +
.../internal/schema/CatalogDescriptorUtils.java | 111 +++++++++++
.../testutils/SchemaToCatalogParamsConverter.java | 208 +++++++++++++++++++++
.../sql/engine/exec/ddl/DdlCommandHandler.java | 2 +-
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 42 ++---
.../exec/ddl/DdlToCatalogCommandConverter.java | 3 +
.../sql/engine/exec/MockedStructuresTest.java | 8 +
.../internal/table/distributed/TableManager.java | 204 +++++++++++++++++++-
.../table/distributed/TableManagerTest.java | 72 ++++---
27 files changed, 812 insertions(+), 275 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index 7823b09d47..d157cbaba2 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -38,6 +38,31 @@ import org.apache.ignite.sql.ColumnType;
* Catalog utils.
*/
public class CatalogUtils {
+ /**
+ * Default TIMESTAMP type precision: microseconds.
+ *
+ * <p>SQL`16 part 2 section 6.1 syntax rule 36
+ */
+ public static final int DEFAULT_TIMESTAMP_PRECISION = 6;
+ /**
+ * Default TIME type precision: seconds.
+ *
+ * <p>SQL`16 part 2 section 6.1 syntax rule 36
+ */
+ public static final int DEFAULT_TIME_PRECISION = 0;
+ /**
+ * Default DECIMAL precision is implementation-defined.
+ *
+ * <p>SQL`16 part 2 section 6.1 syntax rule 25
+ */
+ public static final int DEFAULT_DECIMAL_PRECISION = 19;
+ /**
+ * Default DECIMAL scale is implementation-defined.
+ *
+ * <p>SQL`16 part 2 section 6.1 syntax rule 25
+ */
+ public static final int DEFAULT_DECIMAL_SCALE = 3;
+
private static final Map<ColumnType, Set<ColumnType>> ALTER_COLUMN_TYPE_TRANSITIONS = new EnumMap<>(ColumnType.class);
static {
@@ -125,9 +150,9 @@ public class CatalogUtils {
* @return Column descriptor.
*/
public static CatalogTableColumnDescriptor fromParams(ColumnParams params) {
- int precision = params.precision() != null ? params.precision() : 0;
- int scale = params.scale() != null ? params.scale() : 0;
- int length = params.length() != null ? params.length() : 0;
+ int precision = params.precision() != null ? params.precision() : defaultPrecision(params.type());
+ int scale = params.scale() != null ? params.scale() : defaultScale(params.type());
+ int length = params.length() != null ? params.length() : defaultLength(params.type());
DefaultValue defaultValue = params.defaultValueDefinition();
return new CatalogTableColumnDescriptor(params.name(), params.type(), params.nullable(),
@@ -146,4 +171,42 @@ public class CatalogUtils {
return supportedTransitions != null && supportedTransitions.contains(target);
}
+
+ private static int defaultPrecision(ColumnType columnType) {
+ //TODO IGNITE-19938: Add REAL,FLOAT and DOUBLE precision. See SQL`16 part 2 section 6.1 syntax rule 29-31
+ switch (columnType) {
+ case NUMBER:
+ case DECIMAL:
+ return DEFAULT_DECIMAL_PRECISION;
+ case TIME:
+ return DEFAULT_TIME_PRECISION;
+ case TIMESTAMP:
+ case DATETIME:
+ return DEFAULT_TIMESTAMP_PRECISION;
+ default:
+ return 0;
+ }
+ }
+
+ private static int defaultScale(ColumnType columnType) {
+ //TODO IGNITE-19938: Add REAL,FLOAT and DOUBLE precision. See SQL`16 part 2 section 6.1 syntax rule 29-31
+ if (columnType == ColumnType.DECIMAL) {
+ return DEFAULT_DECIMAL_SCALE;
+ }
+
+ return 0;
+ }
+
+ private static int defaultLength(ColumnType columnType) {
+ //TODO IGNITE-19938: Return length for other types. See SQL`16 part 2 section 6.1 syntax rule 39
+ switch (columnType) {
+ case BITMASK:
+ case STRING:
+ case BYTE_ARRAY:
+ return Integer.MAX_VALUE;
+ default:
+ return 0;
+ }
+ }
+
}
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/ColumnParams.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/ColumnParams.java
index f95e0544d7..9a57887652 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/ColumnParams.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/ColumnParams.java
@@ -158,7 +158,7 @@ public class ColumnParams {
*
* @return {@code this}.
*/
- public Builder precision(int precision) {
+ public Builder precision(Integer precision) {
params.precision = precision;
return this;
@@ -169,7 +169,7 @@ public class ColumnParams {
*
* @return {@code this}.
*/
- public Builder scale(int scale) {
+ public Builder scale(Integer scale) {
params.scale = scale;
return this;
@@ -180,7 +180,7 @@ public class ColumnParams {
*
* @return {@code this}.
*/
- public Builder length(int length) {
+ public Builder length(Integer length) {
params.length = length;
return this;
diff --git a/modules/catalog/src/test/java/org/apache/ignite/internal/catalog/CatalogManagerSelfTest.java b/modules/catalog/src/test/java/org/apache/ignite/internal/catalog/CatalogManagerSelfTest.java
index 44a5e29fa5..bca5bab3f2 100644
--- a/modules/catalog/src/test/java/org/apache/ignite/internal/catalog/CatalogManagerSelfTest.java
+++ b/modules/catalog/src/test/java/org/apache/ignite/internal/catalog/CatalogManagerSelfTest.java
@@ -411,7 +411,7 @@ public class CatalogManagerSelfTest {
assertEquals(DefaultValue.Type.CONSTANT, column.defaultValue().type());
assertEquals("Ignite!", ((DefaultValue.ConstantValue) column.defaultValue()).value());
- assertEquals(0, column.length());
+ assertEquals(Integer.MAX_VALUE, column.length());
assertEquals(0, column.precision());
assertEquals(0, column.scale());
}
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
index 01c6b7c419..dba0049506 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
@@ -125,7 +125,7 @@ import org.apache.ignite.internal.schema.configuration.defaultvalue.ConstantValu
import org.apache.ignite.internal.schema.configuration.defaultvalue.FunctionCallDefaultConfigurationSchema;
import org.apache.ignite.internal.schema.configuration.defaultvalue.NullValueDefaultConfigurationSchema;
import org.apache.ignite.internal.schema.configuration.index.HashIndexConfigurationSchema;
-import org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
import org.apache.ignite.internal.schema.testutils.definition.TableDefinition;
@@ -270,9 +270,7 @@ public class ItRebalanceDistributedTest {
).withPrimaryKey("key").build();
await(nodes.get(0).tableManager.createTableAsync(
- "TBL1",
- ZONE_1_NAME,
- tblChanger -> SchemaConfigurationConverter.convert(schTbl1, tblChanger)
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_1_NAME, schTbl1)
));
assertEquals(1, getPartitionClusterNodes(0, 0).size());
@@ -296,9 +294,7 @@ public class ItRebalanceDistributedTest {
).withPrimaryKey("key").build();
await(nodes.get(0).tableManager.createTableAsync(
- "TBL1",
- ZONE_1_NAME,
- tblChanger -> SchemaConfigurationConverter.convert(schTbl1, tblChanger)
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_1_NAME, schTbl1)
));
assertEquals(1, getPartitionClusterNodes(0, 0).size());
@@ -323,9 +319,8 @@ public class ItRebalanceDistributedTest {
).withPrimaryKey("key").build();
await(nodes.get(0).tableManager.createTableAsync(
- "TBL1",
- ZONE_1_NAME,
- tblChanger -> SchemaConfigurationConverter.convert(schTbl1, tblChanger)));
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_1_NAME, schTbl1)
+ ));
assertEquals(1, getPartitionClusterNodes(0, 0).size());
@@ -353,9 +348,8 @@ public class ItRebalanceDistributedTest {
// Tests that the distribution zone created on node0 is available on node1.
TableImpl table = (TableImpl) await(nodes.get(1).tableManager.createTableAsync(
- "TBL1",
- zoneName,
- tblChanger -> SchemaConfigurationConverter.convert(schTbl1, tblChanger)));
+ SchemaToCatalogParamsConverter.toCreateTable(zoneName, schTbl1)
+ ));
Set<String> partitionNodesConsistentIds = getPartitionClusterNodes(0, 0).stream()
.map(Assignment::consistentId)
@@ -416,9 +410,8 @@ public class ItRebalanceDistributedTest {
).withPrimaryKey("key").build();
await(nodes.get(0).tableManager.createTableAsync(
- "TBL1",
- ZONE_1_NAME,
- tblChanger -> SchemaConfigurationConverter.convert(schTbl1, tblChanger)));
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_1_NAME, schTbl1)
+ ));
assertEquals(1, getPartitionClusterNodes(0, 0).size());
@@ -821,6 +814,7 @@ public class ItRebalanceDistributedTest {
dataStorageMgr,
storagePath,
metaStorageManager,
+ catalogManager,
schemaManager,
view -> new LocalLogStorageFactory(),
new HybridClockImpl(),
@@ -1008,11 +1002,7 @@ public class ItRebalanceDistributedTest {
assertThat(
nodes.get(0).tableManager.createTableAsync(
- tableName,
- zoneName,
- tableChange -> {
- SchemaConfigurationConverter.convert(createTableDefinition(tableName), tableChange);
- }
+ SchemaToCatalogParamsConverter.toCreateTable(zoneName, createTableDefinition(tableName))
),
willCompleteSuccessfully()
);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/inmemory/ItRaftStorageVolatilityTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/inmemory/ItRaftStorageVolatilityTest.java
index 9c7f7d4115..e19cb341e0 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/inmemory/ItRaftStorageVolatilityTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/inmemory/ItRaftStorageVolatilityTest.java
@@ -21,7 +21,6 @@ import static ca.seinesoftware.hamcrest.path.PathMatcher.exists;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_PARTITION_COUNT;
-import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
import static org.apache.ignite.internal.distributionzones.DistributionZonesTestUtil.createZone;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
import static org.hamcrest.MatcherAssert.assertThat;
@@ -42,7 +41,7 @@ import org.apache.ignite.internal.ClusterPerTestIntegrationTest;
import org.apache.ignite.internal.app.IgniteImpl;
import org.apache.ignite.internal.raft.configuration.EntryCountBudgetChange;
import org.apache.ignite.internal.raft.configuration.RaftConfiguration;
-import org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
import org.apache.ignite.internal.schema.testutils.definition.TableDefinition;
@@ -240,7 +239,7 @@ class ItRaftStorageVolatilityTest extends ClusterPerTestIntegrationTest {
}
private void createTableWithMaxOneInMemoryEntryAllowed(String tableName) {
- int zoneId = await(createZone(
+ await(createZone(
node(0).distributionZoneManager(), "zone1", 1, DEFAULT_PARTITION_COUNT,
dataStorageChange -> dataStorageChange.convert(VolatilePageMemoryDataStorageChange.class)));
@@ -249,9 +248,8 @@ class ItRaftStorageVolatilityTest extends ClusterPerTestIntegrationTest {
SchemaBuilders.column("NAME", ColumnType.string()).asNullable(true).build()
).withPrimaryKey("ID").build();
- await(((TableManager) node(0).tables()).createTableAsync(tableName, DEFAULT_ZONE_NAME, tableChange -> {
- SchemaConfigurationConverter.convert(tableDef, tableChange)
- .changeZoneId(zoneId);
- }));
+ await(((TableManager) node(0).tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable("zone1", tableDef)
+ ));
}
}
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/raftsnapshot/ItTableRaftSnapshotsTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/raftsnapshot/ItTableRaftSnapshotsTest.java
index cfc9bc7c9d..0f3069a620 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/raftsnapshot/ItTableRaftSnapshotsTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/raftsnapshot/ItTableRaftSnapshotsTest.java
@@ -354,7 +354,7 @@ class ItTableRaftSnapshotsTest extends IgniteIntegrationTest {
}
private void createTestTableWith3Replicas(String storageEngine) throws InterruptedException {
- String zoneSql = "create zone test_zone"
+ String zoneSql = "create zone TEST_ZONE"
+ (DEFAULT_STORAGE_ENGINE.equals(storageEngine) ? "" : " engine " + storageEngine)
+ " with partitions=1, replicas=3;";
String sql = "create table test (key int primary key, value varchar(20))"
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 831421988c..30dbdd0552 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -153,10 +153,10 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
private static final IntFunction<String> VALUE_PRODUCER = i -> "val " + i;
/** Test table name. */
- private static final String TABLE_NAME = "Table1";
+ private static final String TABLE_NAME = "Table1".toUpperCase();
/** Test table name. */
- private static final String TABLE_NAME_2 = "Table2";
+ private static final String TABLE_NAME_2 = "Table2".toUpperCase();
@InjectConfiguration("mock: " + RAFT_CFG)
private static RaftConfiguration raftConfiguration;
@@ -375,6 +375,7 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
dataStorageManager,
storagePath,
metaStorageMgr,
+ catalogManager,
schemaManager,
view -> new LocalLogStorageFactory(),
hybridClock,
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeKvViewTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeKvViewTest.java
index deb86eaadf..197d44d38c 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeKvViewTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeKvViewTest.java
@@ -28,11 +28,13 @@ import org.apache.ignite.Ignite;
import org.apache.ignite.internal.schema.SchemaMismatchException;
import org.apache.ignite.table.KeyValueView;
import org.apache.ignite.table.Tuple;
+import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
/**
* Ignition interface tests.
*/
+@Disabled("IGNITE-19499")
class ItSchemaChangeKvViewTest extends AbstractSchemaChangeTest {
/**
* Check add a new column to table schema.
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeTableViewTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeTableViewTest.java
index 9906221940..b759356c78 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeTableViewTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItSchemaChangeTableViewTest.java
@@ -35,6 +35,7 @@ import org.junit.jupiter.api.Test;
/**
* Ignition interface tests.
*/
+@Disabled("IGNITE-19499")
class ItSchemaChangeTableViewTest extends AbstractSchemaChangeTest {
/**
* Check add a new column to table schema.
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTableApiContractTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTableApiContractTest.java
index 91da7e94e7..a60da2e800 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTableApiContractTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTableApiContractTest.java
@@ -19,7 +19,6 @@ package org.apache.ignite.internal.runner.app;
import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
-import static org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter.convert;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.contains;
@@ -34,8 +33,14 @@ import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.stream.IntStream;
import org.apache.ignite.Ignite;
+import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.catalog.commands.DefaultValue;
+import org.apache.ignite.internal.catalog.commands.DropTableParams;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
+import org.apache.ignite.internal.schema.testutils.definition.TableDefinition;
import org.apache.ignite.internal.sql.engine.ClusterPerClassIntegrationTest;
import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.lang.TableAlreadyExistsException;
@@ -142,13 +147,19 @@ public class ItTableApiContractTest extends ClusterPerClassIntegrationTest {
*/
@Test
public void testDropTable() throws Exception {
- CompletableFuture<Void> dropTblFut1 = tableManager().dropTableAsync(TABLE_NAME);
+ CompletableFuture<Void> dropTblFut1 = tableManager().dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .build());
dropTblFut1.get();
assertNull(ignite.tables().table(TABLE_NAME));
- CompletableFuture<Void> dropTblFut2 = tableManager().dropTableAsync(TABLE_NAME);
+ CompletableFuture<Void> dropTblFut2 = tableManager().dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .build());
assertThrows(TableNotFoundException.class, () -> futureResult(dropTblFut2));
}
@@ -160,23 +171,29 @@ public class ItTableApiContractTest extends ClusterPerClassIntegrationTest {
*/
@Test
public void testAlterTable() throws Exception {
- await(tableManager().alterTableAsync(TABLE_NAME, chng -> {
- chng.changeColumns(cols ->
- cols.create("NAME_1", colChg -> convert(SchemaBuilders.column("NAME_1", ColumnType.string()).asNullable(true)
- .withDefaultValue("default").build(), colChg)));
- return true;
- }));
+ await(tableManager().alterTableAddColumnAsync(AlterTableAddColumnParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .columns(List.of(
+ ColumnParams.builder().name("NAME_1").type(org.apache.ignite.sql.ColumnType.STRING).nullable(true)
+ .defaultValue(DefaultValue.constant("default")).build()
+ ))
+ .build()
+ ));
assertNotNull(ignite.tables().table(TABLE_NAME));
- assertNull(ignite.tables().table(TABLE_NAME + "_not_exist"));
-
- assertThrows(TableNotFoundException.class, () -> await(tableManager().alterTableAsync(TABLE_NAME + "_not_exist", chng -> {
- chng.changeColumns(cols ->
- cols.create("NAME_1", colChg -> convert(SchemaBuilders.column("NAME_1", ColumnType.string()).asNullable(true)
- .withDefaultValue("default").build(), colChg)));
- return true;
- })));
+ assertNull(ignite.tables().table("UNKNOWN"));
+
+ assertThrows(TableNotFoundException.class, () -> await(tableManager().alterTableAddColumnAsync(AlterTableAddColumnParams.builder()
+ .schemaName(SCHEMA)
+ .tableName("UNKNOWN")
+ .columns(List.of(
+ ColumnParams.builder().name("NAME_1").type(org.apache.ignite.sql.ColumnType.STRING).nullable(true)
+ .defaultValue(DefaultValue.constant("default")).build()
+ ))
+ .build()
+ )));
}
/**
@@ -186,68 +203,49 @@ public class ItTableApiContractTest extends ClusterPerClassIntegrationTest {
*/
@Test
public void testAlterTableAsync() throws Exception {
- CompletableFuture<Void> altTblFut1 = tableManager().alterTableAsync(TABLE_NAME,
- chng -> {
- chng.changeColumns(cols ->
- cols.create("NAME_1", colChg -> convert(SchemaBuilders.column("NAME_1",
- ColumnType.string()).asNullable(true).withDefaultValue("default").build(), colChg)));
- return true;
- });
-
- CompletableFuture<Void> altTblFut2 = tableManager().alterTableAsync(TABLE_NAME + "_not_exist",
- chng -> {
- chng.changeColumns(cols ->
- cols.create("NAME_1", colChg -> convert(SchemaBuilders.column("NAME_1",
- ColumnType.string()).asNullable(true).withDefaultValue("default").build(), colChg)));
- return true;
- });
+ CompletableFuture<Void> altTblFut1 = tableManager().alterTableAddColumnAsync(AlterTableAddColumnParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .columns(List.of(
+ ColumnParams.builder().name("NAME_1").type(org.apache.ignite.sql.ColumnType.STRING).nullable(true)
+ .defaultValue(DefaultValue.constant("default")).build()
+ ))
+ .build());
+
+ CompletableFuture<Void> altTblFut2 = tableManager().alterTableAddColumnAsync(AlterTableAddColumnParams.builder()
+ .schemaName(SCHEMA)
+ .tableName("UNKNOWN")
+ .columns(List.of(
+ ColumnParams.builder().name("NAME_1").type(org.apache.ignite.sql.ColumnType.STRING).nullable(true)
+ .defaultValue(DefaultValue.constant("default")).build()
+ ))
+ .build());
assertNotNull(ignite.tables().table(TABLE_NAME));
- assertNull(ignite.tables().table(TABLE_NAME + "_not_exist"));
+ assertNull(ignite.tables().table("UNKNOWN"));
altTblFut1.get();
assertThrows(TableNotFoundException.class, () -> futureResult(altTblFut2));
}
- /**
- * Checks a contract for table creation.
- *
- * @throws Exception If failed.
- */
- @Test
- public void testCreateTable() throws Exception {
- Table table = ignite.tables().table(TABLE_NAME);
-
- assertNotNull(table);
-
- assertThrows(TableAlreadyExistsException.class,
- () -> await(tableManager().createTableAsync(TABLE_NAME, DEFAULT_ZONE_NAME,
- tableChange -> convert(SchemaBuilders.tableBuilder(SCHEMA, TABLE_NAME)
- .columns(
- SchemaBuilders.column("new_key", ColumnType.INT64).build(),
- SchemaBuilders.column("new_val", ColumnType.string()).build())
- .withPrimaryKey("new_key")
- .build(), tableChange))));
- }
-
/**
* Checks a contract for asynchronous table creation.
- *
- * @throws Exception If failed.
*/
@Test
- public void testCreateTableAsync() throws Exception {
+ public void testCreateTableAsync() {
assertNotNull(ignite.tables().table(TABLE_NAME));
+ TableDefinition tableDefinition = SchemaBuilders.tableBuilder(SCHEMA, TABLE_NAME)
+ .columns(
+ SchemaBuilders.column("new_key", ColumnType.INT64).build(),
+ SchemaBuilders.column("new_val", ColumnType.string()).build())
+ .withPrimaryKey("new_key")
+ .build();
+
CompletableFuture<Table> tableFut2 = tableManager()
- .createTableAsync(TABLE_NAME, DEFAULT_ZONE_NAME, tableChange -> convert(SchemaBuilders.tableBuilder(SCHEMA, TABLE_NAME)
- .columns(
- SchemaBuilders.column("new_key", ColumnType.INT64).build(),
- SchemaBuilders.column("new_val", ColumnType.string()).build())
- .withPrimaryKey("new_key")
- .build(), tableChange));
+ .createTableAsync(SchemaToCatalogParamsConverter.toCreateTable(DEFAULT_ZONE_NAME, tableDefinition));
assertThrows(TableAlreadyExistsException.class, () -> futureResult(tableFut2));
}
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTablesApiTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTablesApiTest.java
index cb96e88300..c4fc440077 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTablesApiTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItTablesApiTest.java
@@ -20,7 +20,6 @@ package org.apache.ignite.internal.runner.app;
import static java.util.concurrent.CompletableFuture.runAsync;
import static java.util.concurrent.CompletableFuture.supplyAsync;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
-import static org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter.convert;
import static org.apache.ignite.internal.test.WatchListenerInhibitor.metastorageEventsInhibitor;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.testNodeName;
@@ -44,6 +43,9 @@ import java.util.stream.IntStream;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgnitionManager;
import org.apache.ignite.InitParameters;
+import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
+import org.apache.ignite.internal.catalog.commands.DropTableParams;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnDefinition;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
@@ -442,7 +444,10 @@ public class ItTablesApiTest extends IgniteAbstractTest {
ignite1Inhibitor.startInhibit();
- await(((TableManager) clusterNodes.get(0).tables()).dropTableAsync(TABLE_NAME));
+ await(((TableManager) clusterNodes.get(0).tables()).dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .build()));
// Because the event inhibitor was started, last metastorage updates do not reach to one node.
// Therefore the table still exists locally, but API prevents getting it.
@@ -466,11 +471,6 @@ public class ItTablesApiTest extends IgniteAbstractTest {
* @param tableName Table name.
*/
protected Table createTable(Ignite node, String tableName) {
- List<ColumnDefinition> cols = new ArrayList<>();
- cols.add(SchemaBuilders.column("key", ColumnType.INT64).build());
- cols.add(SchemaBuilders.column("valInt", ColumnType.INT32).asNullable(true).build());
- cols.add(SchemaBuilders.column("valStr", ColumnType.string()).withDefaultValue("default").build());
-
var tmpl = "CREATE TABLE %s (key BIGINT PRIMARY KEY, valInt INT, valStr VARCHAR)";
var sql = String.format(tmpl, tableName);
@@ -489,17 +489,19 @@ public class ItTablesApiTest extends IgniteAbstractTest {
*/
protected Table createTableIfNotExists(Ignite node, String tableName) {
try {
- return await(((TableManager) node.tables()).createTableAsync(
- tableName,
- DEFAULT_ZONE_NAME,
- tblCh -> convert(SchemaBuilders.tableBuilder(SCHEMA, tableName).columns(Arrays.asList(
+ TableManager tableManager = (TableManager) node.tables();
+
+ await(tableManager.createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(DEFAULT_ZONE_NAME,
+ SchemaBuilders.tableBuilder(SCHEMA, tableName).columns(Arrays.asList(
SchemaBuilders.column("key", ColumnType.INT64).build(),
SchemaBuilders.column("valInt", ColumnType.INT32).asNullable(true).build(),
SchemaBuilders.column("valStr", ColumnType.string())
.withDefaultValue("default").build()
- )).withPrimaryKey("key").build(),
- tblCh)
+ )).withPrimaryKey("key").build())
));
+
+ return tableManager.table(tableName);
} catch (TableAlreadyExistsException ex) {
return node.tables().table(tableName);
}
@@ -512,7 +514,10 @@ public class ItTablesApiTest extends IgniteAbstractTest {
* @param tableName Table name.
*/
protected void dropTable(Ignite node, String tableName) {
- await(((TableManager) node.tables()).dropTableAsync(tableName));
+ await(((TableManager) node.tables()).dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(TABLE_NAME)
+ .build()));
}
/**
@@ -523,7 +528,10 @@ public class ItTablesApiTest extends IgniteAbstractTest {
*/
protected void dropTableIfExists(Ignite node, String tableName) {
try {
- await(((TableManager) node.tables()).dropTableAsync(tableName));
+ await(((TableManager) node.tables()).dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(tableName)
+ .build()));
} catch (TableNotFoundException ex) {
log.info("Dropping the table ignored.", ex);
}
@@ -550,18 +558,12 @@ public class ItTablesApiTest extends IgniteAbstractTest {
* @param colDefinition Column defenition.
*/
private void addColumnInternal(Ignite node, String tableName, ColumnDefinition colDefinition) {
- await(((TableManager) node.tables()).alterTableAsync(
- tableName,
- chng -> {
- chng.changeColumns(cols -> {
- try {
- cols.create(colDefinition.name(), colChg -> convert(colDefinition, colChg));
- } catch (IllegalArgumentException e) {
- throw new ColumnAlreadyExistsException(colDefinition.name());
- }
- });
- return true;
- }));
+ await(((TableManager) node.tables()).alterTableAddColumnAsync(
+ AlterTableAddColumnParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(tableName)
+ .columns(List.of(SchemaToCatalogParamsConverter.convert(colDefinition)))
+ .build()));
}
/**
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
index 67579912c5..c279aa9f62 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/PlatformTestNodeRunner.java
@@ -48,7 +48,7 @@ import org.apache.ignite.internal.schema.SchemaDescriptor;
import org.apache.ignite.internal.schema.marshaller.TupleMarshallerException;
import org.apache.ignite.internal.schema.marshaller.TupleMarshallerImpl;
import org.apache.ignite.internal.schema.row.Row;
-import org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType.TemporalColumnType;
@@ -267,8 +267,8 @@ public class PlatformTestNodeRunner {
SchemaBuilders.column("val", ColumnType.string()).asNullable(true).build()
).withPrimaryKey(keyCol).build();
- await(((TableManager) node.tables()).createTableAsync(schTbl.name(), ZONE_NAME, tblCh ->
- SchemaConfigurationConverter.convert(schTbl, tblCh)
+ await(((TableManager) node.tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, schTbl)
));
int maxTimePrecision = TemporalColumnType.MAX_TIME_PRECISION;
@@ -295,8 +295,8 @@ public class PlatformTestNodeRunner {
SchemaBuilders.column("decimal", ColumnType.decimal()).asNullable(true).build()
).withPrimaryKey(keyCol).build();
- await(((TableManager) node.tables()).createTableAsync(schTblAll.name(), ZONE_NAME, tblCh ->
- SchemaConfigurationConverter.convert(schTblAll, tblCh)
+ await(((TableManager) node.tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, schTblAll)
));
// TODO IGNITE-18431 remove extra table, use TABLE_NAME_ALL_COLUMNS for SQL tests.
@@ -321,8 +321,8 @@ public class PlatformTestNodeRunner {
SchemaBuilders.column("decimal", ColumnType.decimal()).asNullable(true).build()
).withPrimaryKey(keyCol).build();
- await(((TableManager) node.tables()).createTableAsync(schTblAllSql.name(), ZONE_NAME, tblCh ->
- SchemaConfigurationConverter.convert(schTblAllSql, tblCh)
+ await(((TableManager) node.tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, schTblAllSql)
));
createTwoColumnTable(node, ColumnType.INT8);
@@ -351,8 +351,8 @@ public class PlatformTestNodeRunner {
SchemaBuilders.column("val", type).asNullable(true).build()
).withPrimaryKey(keyCol).build();
- await(((TableManager) node.tables()).createTableAsync(schTbl.name(), ZONE_NAME, tblCh ->
- SchemaConfigurationConverter.convert(schTbl, tblCh)
+ await(((TableManager) node.tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, schTbl)
));
}
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItCommonApiTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItCommonApiTest.java
index 2211137be9..0c1d3b55e9 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItCommonApiTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItCommonApiTest.java
@@ -33,7 +33,7 @@ import java.util.concurrent.TimeUnit;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.ignite.Ignite;
import org.apache.ignite.internal.app.IgniteImpl;
-import org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType.TemporalColumnType;
@@ -144,8 +144,8 @@ public class ItCommonApiTest extends ClusterPerClassIntegrationTest {
SchemaBuilders.column("datetime", ColumnType.datetime(maxTimePrecision)).asNullable(true).build()
).withPrimaryKey(keyCol).build();
- await(((TableManager) node.tables()).createTableAsync(schTblAllSql.name(), DEFAULT_ZONE_NAME, tblCh ->
- SchemaConfigurationConverter.convert(schTblAllSql, tblCh)
+ await(((TableManager) node.tables()).createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(DEFAULT_ZONE_NAME, schTblAllSql)
));
Table tbl = node.tables().table(kvTblName);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlAsynchronousApiTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlAsynchronousApiTest.java
index 473d6399f7..105463070c 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlAsynchronousApiTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlAsynchronousApiTest.java
@@ -22,7 +22,6 @@ import static org.apache.ignite.internal.sql.engine.util.QueryChecker.containsTa
import static org.apache.ignite.internal.testframework.IgniteTestUtils.assertThrowsWithCause;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
import static org.apache.ignite.lang.ErrorGroups.Sql.CONSTRAINT_VIOLATION_ERR;
-import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
@@ -182,26 +181,21 @@ public class ItSqlAsynchronousApiTest extends ClusterPerClassIntegrationTest {
checkError(
SqlException.class,
- "Can`t delete column(s). Column VAL1 is used by indexes [TEST_IDX3].",
+ "Can't drop indexed column: [columnName=VAL1, indexName=TEST_IDX3]",
ses,
"ALTER TABLE TEST DROP COLUMN val1"
);
- SqlException ex = IgniteTestUtils.cause(assertThrows(Throwable.class,
- () -> await(ses.executeAsync(null, "ALTER TABLE TEST DROP COLUMN (val0, val1)"))), SqlException.class);
- assertNotNull(ex);
- assertEquals(STMT_VALIDATION_ERR, ex.code());
-
- String msg = ex.getMessage();
- String explainMsg = "Unexpected error message: " + msg;
-
- assertTrue(msg.contains("Column VAL0 is used by indexes ["), explainMsg);
- assertTrue(msg.contains("TEST_IDX1") && msg.contains("TEST_IDX2") && msg.contains("TEST_IDX3"), explainMsg);
- assertTrue(msg.contains("Column VAL1 is used by indexes [TEST_IDX3]"), explainMsg);
+ checkError(
+ SqlException.class,
+ "Can't drop indexed column: [columnName=VAL0, indexName=TEST_IDX1]",
+ ses,
+ "ALTER TABLE TEST DROP COLUMN (val0, val1)"
+ );
checkError(
SqlException.class,
- "Can`t delete column, belongs to primary key: [name=ID]",
+ "Can't drop primary key column: [name=ID]",
ses,
"ALTER TABLE TEST DROP COLUMN id"
);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlSynchronousApiTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlSynchronousApiTest.java
index 376f71f63f..f2c713c34c 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlSynchronousApiTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/api/ItSqlSynchronousApiTest.java
@@ -18,11 +18,8 @@
package org.apache.ignite.internal.sql.api;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.assertThrowsWithCause;
-import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
-import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
-import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -165,26 +162,21 @@ public class ItSqlSynchronousApiTest extends ClusterPerClassIntegrationTest {
checkError(
SqlException.class,
- "Can`t delete column(s). Column VAL1 is used by indexes [TEST_IDX3].",
+ "Can't drop indexed column: [columnName=VAL1, indexName=TEST_IDX3]",
ses,
"ALTER TABLE TEST DROP COLUMN val1"
);
- SqlException ex = IgniteTestUtils.cause(assertThrows(Throwable.class,
- () -> await(ses.executeAsync(null, "ALTER TABLE TEST DROP COLUMN (val0, val1)"))), SqlException.class);
- assertNotNull(ex);
- assertEquals(STMT_VALIDATION_ERR, ex.code());
-
- String msg = ex.getMessage();
- String explainMsg = "Unexpected error message: " + msg;
-
- assertTrue(msg.contains("Column VAL0 is used by indexes ["), explainMsg);
- assertTrue(msg.contains("TEST_IDX1") && msg.contains("TEST_IDX2") && msg.contains("TEST_IDX3"), explainMsg);
- assertTrue(msg.contains("Column VAL1 is used by indexes [TEST_IDX3]"), explainMsg);
+ checkError(
+ SqlException.class,
+ "Can't drop indexed column: [columnName=VAL0, indexName=TEST_IDX1]",
+ ses,
+ "ALTER TABLE TEST DROP COLUMN (val0, val1)"
+ );
checkError(
SqlException.class,
- "Can`t delete column, belongs to primary key: [name=ID]",
+ "Can't drop primary key column: [name=ID]",
ses,
"ALTER TABLE TEST DROP COLUMN id"
);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/engine/ItCreateTableDdlTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/engine/ItCreateTableDdlTest.java
index 6f81d7e757..d292d3246f 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/engine/ItCreateTableDdlTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/sql/engine/ItCreateTableDdlTest.java
@@ -25,13 +25,8 @@ import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.util.List;
-import org.apache.ignite.Ignite;
-import org.apache.ignite.internal.configuration.ConfigurationManager;
import org.apache.ignite.internal.schema.Column;
-import org.apache.ignite.internal.schema.configuration.ExtendedTableView;
-import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.table.TableImpl;
-import org.apache.ignite.internal.testframework.IgniteTestUtils;
import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.sql.SqlException;
import org.junit.jupiter.api.AfterEach;
@@ -150,7 +145,7 @@ public class ItCreateTableDdlTest extends ClusterPerClassIntegrationTest {
IgniteException.class,
() -> sql("CREATE TABLE T0(ID0 INT, ID1 INT, VAL INT, PRIMARY KEY (ID1, ID0)) COLOCATE (ID1, ID0, ID1)")
).getMessage(),
- containsString("Colocation columns contains duplicates: [duplicates=[ID1]]]")
+ containsString("Colocation columns contains duplicates: ID1, ID0, ID1")
);
}
@@ -202,29 +197,6 @@ public class ItCreateTableDdlTest extends ClusterPerClassIntegrationTest {
assertEquals(3, res.get(0).get(0));
}
- /**
- * Checks that schema version is updated even if column names are intersected.
- */
- // Need to be removed after https://issues.apache.org/jira/browse/IGNITE-19082
- @Test
- public void checkSchemaUpdatedWithEqAlterColumn() {
- sql("CREATE TABLE TEST(ID INT PRIMARY KEY, VAL0 INT)");
-
- Ignite node = CLUSTER_NODES.get(0);
-
- ConfigurationManager cfgMgr = IgniteTestUtils.getFieldValue(node, "clusterCfgMgr");
-
- TablesConfiguration tablesConfiguration = cfgMgr.configurationRegistry().getConfiguration(TablesConfiguration.KEY);
-
- int schIdBefore = ((ExtendedTableView) tablesConfiguration.tables().get("TEST").value()).schemaId();
-
- sql("ALTER TABLE TEST ADD COLUMN (VAL1 INT)");
-
- int schIdAfter = ((ExtendedTableView) tablesConfiguration.tables().get("TEST").value()).schemaId();
-
- assertEquals(schIdBefore + 1, schIdAfter);
- }
-
/**
* Check explicit colocation columns configuration.
*/
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/table/ItRoReadsTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/table/ItRoReadsTest.java
index ff2a3b83f1..55bf5008eb 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/table/ItRoReadsTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/table/ItRoReadsTest.java
@@ -20,7 +20,6 @@ package org.apache.ignite.internal.table;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_REPLICA_COUNT;
import static org.apache.ignite.internal.distributionzones.DistributionZonesTestUtil.createZone;
import static org.apache.ignite.internal.runner.app.ItTablesApiTest.SCHEMA;
-import static org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter.convert;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.testNodeName;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
@@ -43,6 +42,7 @@ import org.apache.ignite.Ignite;
import org.apache.ignite.IgnitionManager;
import org.apache.ignite.InitParameters;
import org.apache.ignite.internal.app.IgniteImpl;
+import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.logger.IgniteLogger;
import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.schema.BinaryRow;
@@ -52,6 +52,7 @@ import org.apache.ignite.internal.schema.NativeTypes;
import org.apache.ignite.internal.schema.SchemaDescriptor;
import org.apache.ignite.internal.schema.row.Row;
import org.apache.ignite.internal.schema.row.RowAssembler;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnDefinition;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
@@ -82,7 +83,7 @@ import org.junit.jupiter.api.extension.ExtendWith;
public class ItRoReadsTest extends BaseIgniteAbstractTest {
private static final IgniteLogger LOG = Loggers.forClass(ItRoReadsTest.class);
- private static final String TABLE_NAME = "some-table";
+ private static final String TABLE_NAME = "some-table".toUpperCase();
private static final SchemaDescriptor SCHEMA_1 = new SchemaDescriptor(
1,
@@ -522,18 +523,23 @@ public class ItRoReadsTest extends BaseIgniteAbstractTest {
cols.add(SchemaBuilders.column("valStr", ColumnType.string()).withDefaultValue("default").build());
String zoneName = "zone_" + tableName;
- int zoneId = await(createZone(((IgniteImpl) node).distributionZoneManager(), zoneName, 1, DEFAULT_REPLICA_COUNT));
+ await(createZone(((IgniteImpl) node).distributionZoneManager(), zoneName, 1, DEFAULT_REPLICA_COUNT));
- return await(((TableManager) node.tables()).createTableAsync(
- tableName,
- zoneName,
- tblCh -> convert(SchemaBuilders.tableBuilder(SCHEMA, tableName).columns(
- cols).withPrimaryKey("key").build(), tblCh)
+ TableManager tableManager = (TableManager) node.tables();
+
+ await(tableManager.createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(zoneName, SchemaBuilders.tableBuilder(SCHEMA, tableName).columns(
+ cols).withPrimaryKey("key").build())
));
+
+ return tableManager.table(tableName);
}
private static void stopTable(Ignite node, String tableName) {
- await(((TableManager) node.tables()).dropTableAsync(tableName));
+ await(((TableManager) node.tables()).dropTableAsync(DropTableParams.builder()
+ .schemaName(SCHEMA)
+ .tableName(tableName)
+ .build()));
await(((IgniteImpl) node).distributionZoneManager().dropZone("zone_" + tableName));
}
diff --git a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
index 94c63b5d6e..d38a1597e8 100644
--- a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
+++ b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
@@ -539,6 +539,7 @@ public class IgniteImpl implements Ignite {
dataStorageMgr,
storagePath,
metaStorageMgr,
+ catalogManager,
schemaManager,
volatileLogStorageFactoryCreator,
clock,
diff --git a/modules/schema/build.gradle b/modules/schema/build.gradle
index 92e8100a45..088019e4e4 100644
--- a/modules/schema/build.gradle
+++ b/modules/schema/build.gradle
@@ -52,6 +52,7 @@ dependencies {
testFixturesImplementation project(':ignite-api')
testFixturesImplementation project(':ignite-bytecode')
testFixturesImplementation project(':ignite-configuration-api')
+ testFixturesImplementation project(':ignite-catalog')
testFixturesImplementation(testFixtures(project(':ignite-core')))
testFixturesImplementation libs.jetbrains.annotations
testFixturesImplementation libs.junit5.api
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
index 8acc7aa194..54dca3f1b1 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
@@ -22,7 +22,11 @@ import static org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollat
import static org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation.DESC_NULLS_FIRST;
import java.util.List;
+import org.apache.ignite.internal.catalog.commands.CatalogUtils;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
import org.apache.ignite.internal.catalog.commands.DefaultValue;
+import org.apache.ignite.internal.catalog.commands.DefaultValue.ConstantValue;
+import org.apache.ignite.internal.catalog.commands.DefaultValue.FunctionCall;
import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
import org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
@@ -30,6 +34,8 @@ import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogSortedIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableColumnDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
+import org.apache.ignite.internal.schema.configuration.ColumnChange;
+import org.apache.ignite.internal.schema.configuration.ColumnTypeChange;
import org.apache.ignite.internal.schema.configuration.ColumnTypeView;
import org.apache.ignite.internal.schema.configuration.ColumnView;
import org.apache.ignite.internal.schema.configuration.ConfigurationToSchemaDescriptorConverter;
@@ -38,8 +44,11 @@ import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.ValueSerializationHelper;
import org.apache.ignite.internal.schema.configuration.defaultvalue.ColumnDefaultConfigurationSchema;
import org.apache.ignite.internal.schema.configuration.defaultvalue.ColumnDefaultView;
+import org.apache.ignite.internal.schema.configuration.defaultvalue.ConstantValueDefaultChange;
import org.apache.ignite.internal.schema.configuration.defaultvalue.ConstantValueDefaultView;
+import org.apache.ignite.internal.schema.configuration.defaultvalue.FunctionCallDefaultChange;
import org.apache.ignite.internal.schema.configuration.defaultvalue.FunctionCallDefaultView;
+import org.apache.ignite.internal.schema.configuration.defaultvalue.NullValueDefaultChange;
import org.apache.ignite.internal.schema.configuration.index.HashIndexView;
import org.apache.ignite.internal.schema.configuration.index.IndexColumnView;
import org.apache.ignite.internal.schema.configuration.index.SortedIndexView;
@@ -197,4 +206,106 @@ public class CatalogDescriptorUtils {
return new CatalogIndexColumnDescriptor(config.name(), collation);
}
+
+ /**
+ * Applies changes from params to the configuraiton changer.
+ *
+ * @param params Column change parameters.
+ * @param columnChange Configuration changer.
+ */
+ public static void convertColumnDefinition(ColumnParams params, ColumnChange columnChange) {
+ NativeType nativeType = getNativeType(CatalogUtils.fromParams(params));
+
+ columnChange.changeType(columnTypeChange -> convert(nativeType, columnTypeChange));
+ columnChange.changeNullable(params.nullable());
+ columnChange.changeDefaultValueProvider(defaultChange -> {
+ switch (params.defaultValueDefinition().type()) {
+ case CONSTANT:
+ ConstantValue constantValue = params.defaultValueDefinition();
+
+ var val = constantValue.value();
+
+ if (val != null) {
+ defaultChange.convert(ConstantValueDefaultChange.class)
+ .changeDefaultValue(ValueSerializationHelper.toString(val, nativeType));
+ } else {
+ defaultChange.convert(NullValueDefaultChange.class);
+ }
+
+ break;
+ case FUNCTION_CALL:
+ FunctionCall functionCall = params.defaultValueDefinition();
+
+ defaultChange.convert(FunctionCallDefaultChange.class)
+ .changeFunctionName(functionCall.functionName());
+
+ break;
+ default:
+ throw new IllegalStateException("Unknown default value definition type [type="
+ + params.defaultValueDefinition().type() + ']');
+ }
+ });
+ }
+
+ private static void convert(NativeType colType, ColumnTypeChange colTypeChg) {
+ NativeTypeSpec spec = colType.spec();
+ String typeName = spec.name().toUpperCase();
+
+ colTypeChg.changeType(typeName);
+
+ switch (spec) {
+ case INT8:
+ case INT16:
+ case INT32:
+ case INT64:
+ case FLOAT:
+ case DOUBLE:
+ case DATE:
+ case UUID:
+ // do nothing
+ break;
+
+ case BITMASK:
+ BitmaskNativeType bitmaskColType = (BitmaskNativeType) colType;
+
+ colTypeChg.changeLength(bitmaskColType.bits());
+
+ break;
+
+ case BYTES:
+ case STRING:
+ VarlenNativeType varLenColType = (VarlenNativeType) colType;
+
+ colTypeChg.changeLength(varLenColType.length());
+
+ break;
+
+ case DECIMAL:
+ DecimalNativeType numColType = (DecimalNativeType) colType;
+
+ colTypeChg.changePrecision(numColType.precision());
+ colTypeChg.changeScale(numColType.scale());
+
+ break;
+
+ case NUMBER:
+ NumberNativeType numType = (NumberNativeType) colType;
+
+ colTypeChg.changePrecision(numType.precision());
+
+ break;
+
+ case TIME:
+ case DATETIME:
+ case TIMESTAMP:
+ TemporalNativeType temporalColType = (TemporalNativeType) colType;
+
+ colTypeChg.changePrecision(temporalColType.precision());
+
+ break;
+
+ default:
+ throw new IllegalArgumentException("Unknown type " + colType.spec().name());
+ }
+ }
}
diff --git a/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java b/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java
new file mode 100644
index 0000000000..a00eda8c7f
--- /dev/null
+++ b/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java
@@ -0,0 +1,208 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.schema.testutils;
+
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.catalog.commands.ColumnParams.Builder;
+import org.apache.ignite.internal.catalog.commands.CreateTableParams;
+import org.apache.ignite.internal.catalog.commands.DefaultValue;
+import org.apache.ignite.internal.schema.testutils.definition.ColumnDefinition;
+import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
+import org.apache.ignite.internal.schema.testutils.definition.DefaultValueDefinition.ConstantValue;
+import org.apache.ignite.internal.schema.testutils.definition.DefaultValueDefinition.FunctionCall;
+import org.apache.ignite.internal.schema.testutils.definition.TableDefinition;
+
+/**
+ * Schema to Catalog params converter.
+ */
+public class SchemaToCatalogParamsConverter {
+ /**
+ * Types map.
+ */
+ private static final EnumSet<ColumnType.ColumnTypeSpec> fixSizedTypes = EnumSet.of(
+ ColumnType.INT8.typeSpec(),
+ ColumnType.INT16.typeSpec(),
+ ColumnType.INT32.typeSpec(),
+ ColumnType.INT64.typeSpec(),
+ ColumnType.FLOAT.typeSpec(),
+ ColumnType.DOUBLE.typeSpec(),
+ ColumnType.UUID.typeSpec(),
+ ColumnType.DATE.typeSpec()
+ );
+
+ /**
+ * Converts table definition to CreateTableParams.
+ */
+ public static CreateTableParams toCreateTable(String zoneName, TableDefinition tableDef) {
+ Set<String> keyColumns = tableDef.keyColumns();
+
+ List<String> keyColumnsInOrder = tableDef.columns().stream()
+ .map(ColumnDefinition::name)
+ .filter(keyColumns::contains)
+ .collect(Collectors.toList());
+
+ List<ColumnParams> columnParams = tableDef.columns().stream()
+ .map(SchemaToCatalogParamsConverter::convert)
+ .collect(Collectors.toList());
+
+ return CreateTableParams.builder()
+ .schemaName(tableDef.schemaName())
+ .tableName(tableDef.name())
+ .zone(zoneName)
+ .columns(columnParams)
+ .primaryKeyColumns(keyColumnsInOrder)
+ .colocationColumns(tableDef.colocationColumns())
+ .build();
+ }
+
+ /**
+ * Convert column to column change.
+ *
+ * @param def Column to convert.
+ * @return Column params.
+ */
+ public static ColumnParams convert(ColumnDefinition def) {
+ Builder builder = ColumnParams.builder()
+ .name(def.name())
+ .nullable(def.nullable());
+
+ setType(builder, def.type());
+
+ assert def.defaultValueDefinition() != null;
+
+ switch (def.defaultValueDefinition().type()) {
+ case CONSTANT:
+ ConstantValue constantValue = def.defaultValueDefinition();
+
+ builder.defaultValue(DefaultValue.constant(constantValue.value()));
+
+ break;
+ case FUNCTION_CALL:
+ FunctionCall functionCall = def.defaultValueDefinition();
+
+ builder.defaultValue(DefaultValue.functionCall(functionCall.functionName()));
+
+ break;
+ case NULL:
+
+ builder.defaultValue(DefaultValue.constant(null));
+ break;
+ default:
+ throw new IllegalStateException("Unknown default value definition type [type="
+ + def.defaultValueDefinition().type() + ']');
+ }
+
+ return builder.build();
+ }
+
+ private static org.apache.ignite.sql.ColumnType convert(ColumnType colType) {
+ switch (colType.typeSpec()) {
+ case INT8:
+ return org.apache.ignite.sql.ColumnType.INT8;
+ case INT16:
+ return org.apache.ignite.sql.ColumnType.INT16;
+ case INT32:
+ return org.apache.ignite.sql.ColumnType.INT32;
+ case INT64:
+ return org.apache.ignite.sql.ColumnType.INT64;
+ case FLOAT:
+ return org.apache.ignite.sql.ColumnType.FLOAT;
+ case DOUBLE:
+ return org.apache.ignite.sql.ColumnType.DOUBLE;
+ case DECIMAL:
+ return org.apache.ignite.sql.ColumnType.DECIMAL;
+ case DATE:
+ return org.apache.ignite.sql.ColumnType.DATE;
+ case TIME:
+ return org.apache.ignite.sql.ColumnType.TIME;
+ case DATETIME:
+ return org.apache.ignite.sql.ColumnType.DATETIME;
+ case TIMESTAMP:
+ return org.apache.ignite.sql.ColumnType.TIMESTAMP;
+ case UUID:
+ return org.apache.ignite.sql.ColumnType.UUID;
+ case BITMASK:
+ return org.apache.ignite.sql.ColumnType.BITMASK;
+ case STRING:
+ return org.apache.ignite.sql.ColumnType.STRING;
+ case BYTES:
+ return org.apache.ignite.sql.ColumnType.BYTE_ARRAY;
+ case NUMBER:
+ return org.apache.ignite.sql.ColumnType.NUMBER;
+
+ default:
+ throw new IllegalArgumentException("Type is not supported: " + colType.typeSpec());
+ }
+ }
+
+ /**
+ * Convert ColumnType to ColumnTypeChange.
+ *
+ * @param builder Column builder.
+ * @param colType ColumnType.
+ */
+ private static void setType(Builder builder, ColumnType colType) {
+ builder.type(convert(colType));
+
+ if (fixSizedTypes.contains(colType.typeSpec())) {
+ return;
+ }
+
+ switch (colType.typeSpec()) {
+ case BITMASK:
+ case BYTES:
+ case STRING:
+ int length = ((ColumnType.VarLenColumnType) colType).length();
+
+ if (length == 0) {
+ length = Integer.MAX_VALUE;
+ }
+
+ builder.length(length);
+
+ return;
+ case DECIMAL:
+ ColumnType.DecimalColumnType numColType = (ColumnType.DecimalColumnType) colType;
+
+ builder.precision(numColType.precision());
+ builder.scale(numColType.scale());
+
+ return;
+ case NUMBER:
+ ColumnType.NumberColumnType numType = (ColumnType.NumberColumnType) colType;
+
+ builder.precision(numType.precision());
+
+ return;
+ case TIME:
+ case DATETIME:
+ case TIMESTAMP:
+ ColumnType.TemporalColumnType temporalColType = (ColumnType.TemporalColumnType) colType;
+
+ builder.precision(temporalColType.precision());
+
+ return;
+ default:
+ throw new IllegalArgumentException("Unknown type " + colType.typeSpec().name());
+ }
+ }
+}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index 0bc672a58e..f56848c215 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -104,7 +104,7 @@ import org.apache.ignite.sql.SqlException;
public class DdlCommandHandler {
private final DistributionZoneManager distributionZoneManager;
- private final TableManager tableManager;
+ protected final TableManager tableManager;
private final IndexManager indexManager;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
index a7a9b7f78e..bb702d4566 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
@@ -66,40 +66,32 @@ public class DdlCommandHandlerWrapper extends DdlCommandHandler {
/** Handles ddl commands. */
@Override
public CompletableFuture<Boolean> handle(DdlCommand cmd) {
- // Handle command in usual way.
- CompletableFuture<Boolean> ddlCommandFuture = super.handle(cmd);
-
- // Pass supported commands to the Catalog.
+ // TODO IGNITE-19082 replace TableManager with CatalogManager calls.
if (cmd instanceof CreateTableCommand) {
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.createTable(DdlToCatalogCommandConverter.convert((CreateTableCommand) cmd))
- .handle(handleModificationResult(((CreateTableCommand) cmd).ifTableExists(), TableAlreadyExistsException.class))
- );
+ return tableManager.createTableAsync(DdlToCatalogCommandConverter.convert((CreateTableCommand) cmd))
+ .handle(handleModificationResult(((CreateTableCommand) cmd).ifTableExists(), TableAlreadyExistsException.class));
} else if (cmd instanceof DropTableCommand) {
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.dropTable(DdlToCatalogCommandConverter.convert((DropTableCommand) cmd))
- .handle(handleModificationResult(((DropTableCommand) cmd).ifTableExists(), TableNotFoundException.class))
- );
+ return tableManager.dropTableAsync(DdlToCatalogCommandConverter.convert((DropTableCommand) cmd))
+ .handle(handleModificationResult(((DropTableCommand) cmd).ifTableExists(), TableNotFoundException.class));
} else if (cmd instanceof AlterTableAddCommand) {
AlterTableAddCommand addCommand = (AlterTableAddCommand) cmd;
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.addColumn(DdlToCatalogCommandConverter.convert(addCommand))
- .handle(handleModificationResult(addCommand.ifTableExists(), TableNotFoundException.class))
- );
+ return tableManager.alterTableAddColumnAsync(DdlToCatalogCommandConverter.convert(addCommand))
+ .handle(handleModificationResult(addCommand.ifTableExists(), TableNotFoundException.class));
} else if (cmd instanceof AlterTableDropCommand) {
AlterTableDropCommand dropCommand = (AlterTableDropCommand) cmd;
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.dropColumn(DdlToCatalogCommandConverter.convert(dropCommand))
- .handle(handleModificationResult(dropCommand.ifTableExists(), TableNotFoundException.class))
- );
+ return tableManager.alterTableDropColumnAsync(DdlToCatalogCommandConverter.convert(dropCommand))
+ .handle(handleModificationResult(dropCommand.ifTableExists(), TableNotFoundException.class));
} else if (cmd instanceof AlterColumnCommand) {
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.alterColumn(DdlToCatalogCommandConverter.convert((AlterColumnCommand) cmd))
- .handle(handleModificationResult(((AlterColumnCommand) cmd).ifTableExists(), TableNotFoundException.class))
- );
- } else if (cmd instanceof CreateIndexCommand) {
+ return catalogManager.alterColumn(DdlToCatalogCommandConverter.convert((AlterColumnCommand) cmd))
+ .handle(handleModificationResult(((AlterColumnCommand) cmd).ifTableExists(), TableNotFoundException.class));
+ }
+
+ // Handle command in usual way.
+ CompletableFuture<Boolean> ddlCommandFuture = super.handle(cmd);
+
+ if (cmd instanceof CreateIndexCommand) {
return ddlCommandFuture
.thenCompose(res -> {
AbstractIndexCommandParams params = DdlToCatalogCommandConverter.convert((CreateIndexCommand) cmd);
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
index 22eb89195b..3e958d982b 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlToCatalogCommandConverter.java
@@ -216,6 +216,9 @@ class DdlToCatalogCommandConverter {
.name(def.name())
.type(TypeUtils.columnType(def.type()))
.nullable(def.nullable())
+ .precision(def.precision())
+ .scale(def.scale())
+ .length(def.precision())
.defaultValue(convert(def.defaultValueDefinition()))
.build();
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index ab52605ca2..9aa9471037 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -31,6 +31,8 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.when;
@@ -46,6 +48,7 @@ import java.util.function.Consumer;
import java.util.function.LongFunction;
import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologySnapshot;
import org.apache.ignite.internal.configuration.ConfigurationRegistry;
@@ -279,9 +282,13 @@ public class MockedStructuresTest extends IgniteAbstractTest {
schemaManager.start();
+ //TODO IGNITE-19082 drop mocked catalog manager.
catalogManager = mock(CatalogManager.class);
+ CatalogTableDescriptor descriptor = mock(CatalogTableDescriptor.class);
+ when(descriptor.id()).thenReturn(1);
when(catalogManager.createTable(any())).thenReturn(completedFuture(null));
when(catalogManager.dropTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.table(anyString(), anyLong())).thenReturn(descriptor);
cmgMgr = mock(ClusterManagementGroupManager.class);
@@ -585,6 +592,7 @@ public class MockedStructuresTest extends IgniteAbstractTest {
dataStorageManager,
workDir,
msm,
+ catalogManager,
schemaManager,
null,
clock,
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 72f9ad055c..375a1fa542 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -25,6 +25,8 @@ import static java.util.concurrent.CompletableFuture.failedFuture;
import static java.util.concurrent.CompletableFuture.runAsync;
import static java.util.concurrent.CompletableFuture.supplyAsync;
import static java.util.stream.Collectors.toList;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
+import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_ZONE_NAME;
import static org.apache.ignite.internal.causality.IncrementalVersionedValue.dependingOn;
import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.getZoneById;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.partitionAssignments;
@@ -34,6 +36,7 @@ import static org.apache.ignite.internal.metastorage.dsl.Operations.put;
import static org.apache.ignite.internal.schema.CatalogDescriptorUtils.toTableDescriptor;
import static org.apache.ignite.internal.schema.SchemaManager.INITIAL_SCHEMA_VERSION;
import static org.apache.ignite.internal.schema.configuration.SchemaConfigurationUtils.findTableView;
+import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
import static org.apache.ignite.internal.util.IgniteUtils.shutdownAndAwaitTermination;
import static org.apache.ignite.internal.utils.RebalanceUtil.ASSIGNMENTS_SWITCH_REDUCE_PREFIX;
@@ -43,6 +46,7 @@ import static org.apache.ignite.internal.utils.RebalanceUtil.extractPartitionNum
import static org.apache.ignite.internal.utils.RebalanceUtil.extractTableId;
import static org.apache.ignite.internal.utils.RebalanceUtil.pendingPartAssignmentsKey;
import static org.apache.ignite.internal.utils.RebalanceUtil.stablePartAssignmentsKey;
+import static org.apache.ignite.lang.ErrorGroups.Sql.DROP_IDX_COLUMN_CONSTRAINT_ERR;
import it.unimi.dsi.fastutil.ints.Int2ObjectOpenHashMap;
import java.io.IOException;
@@ -88,6 +92,12 @@ import org.apache.ignite.configuration.notifications.ConfigurationNotificationEv
import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.affinity.Assignment;
import org.apache.ignite.internal.baseline.BaselineManager;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
+import org.apache.ignite.internal.catalog.commands.AlterTableDropColumnParams;
+import org.apache.ignite.internal.catalog.commands.ColumnParams;
+import org.apache.ignite.internal.catalog.commands.CreateTableParams;
+import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
@@ -125,10 +135,12 @@ import org.apache.ignite.internal.raft.storage.impl.LogStorageFactoryCreator;
import org.apache.ignite.internal.replicator.ReplicaManager;
import org.apache.ignite.internal.replicator.ReplicaService;
import org.apache.ignite.internal.replicator.TablePartitionId;
+import org.apache.ignite.internal.schema.CatalogDescriptorUtils;
import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.SchemaRegistry;
import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
+import org.apache.ignite.internal.schema.configuration.PrimaryKeyView;
import org.apache.ignite.internal.schema.configuration.TableChange;
import org.apache.ignite.internal.schema.configuration.TableConfiguration;
import org.apache.ignite.internal.schema.configuration.TableView;
@@ -184,6 +196,8 @@ import org.apache.ignite.internal.util.PendingComparableValuesTracker;
import org.apache.ignite.internal.utils.RebalanceUtil;
import org.apache.ignite.internal.vault.VaultManager;
import org.apache.ignite.lang.ByteArray;
+import org.apache.ignite.lang.ColumnAlreadyExistsException;
+import org.apache.ignite.lang.ColumnNotFoundException;
import org.apache.ignite.lang.DistributionZoneNotFoundException;
import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteExceptionUtils;
@@ -198,6 +212,7 @@ import org.apache.ignite.network.ClusterService;
import org.apache.ignite.network.MessagingService;
import org.apache.ignite.network.TopologyService;
import org.apache.ignite.raft.jraft.storage.impl.VolatileRaftMetaStorage;
+import org.apache.ignite.sql.SqlException;
import org.apache.ignite.table.Table;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
@@ -207,8 +222,6 @@ import org.jetbrains.annotations.TestOnly;
* Table manager.
*/
public class TableManager extends Producer<TableEvent, TableEventParameters> implements IgniteTablesInternal, IgniteComponent {
- private static final String DEFAULT_SCHEMA_NAME = "PUBLIC";
-
private static final long QUERY_DATA_NODES_COUNT_TIMEOUT = TimeUnit.SECONDS.toMillis(3);
/** The logger. */
@@ -315,6 +328,9 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
/** Schema manager. */
private final SchemaManager schemaManager;
+ /** Catalog manager. */
+ private final CatalogManager catalogManager;
+
private final LogStorageFactoryCreator volatileLogStorageFactoryCreator;
/** Executor for scheduling retries of a rebalance. */
@@ -390,6 +406,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param baselineMgr Baseline manager.
* @param txManager Transaction manager.
* @param dataStorageMgr Data storage manager.
+ * @param catalogManager Catalog manager.
* @param schemaManager Schema manager.
* @param volatileLogStorageFactoryCreator Creator for {@link org.apache.ignite.internal.raft.storage.LogStorageFactory} for
* volatile tables.
@@ -413,6 +430,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
DataStorageManager dataStorageMgr,
Path storagePath,
MetaStorageManager metaStorageMgr,
+ CatalogManager catalogManager,
SchemaManager schemaManager,
LogStorageFactoryCreator volatileLogStorageFactoryCreator,
HybridClock clock,
@@ -437,6 +455,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
this.metaStorageMgr = metaStorageMgr;
this.vaultManager = vaultManager;
this.schemaManager = schemaManager;
+ this.catalogManager = catalogManager;
this.volatileLogStorageFactoryCreator = volatileLogStorageFactoryCreator;
this.clock = clock;
this.outgoingSnapshotsManager = outgoingSnapshotsManager;
@@ -1343,7 +1362,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
/**
- * Completes appropriate future to return result from API {@link TableManager#createTableAsync(String, String, Consumer)}.
+ * Completes appropriate future to return result from API {@link TableManager#createTableAsync(CreateTableParams)}.
*
* @param table Table.
*/
@@ -1473,19 +1492,59 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* </ul>
* @see TableAlreadyExistsException
*/
+ @Deprecated
public CompletableFuture<Table> createTableAsync(String name, String zoneName, Consumer<TableChange> tableInitChange) {
+ throw new UnsupportedOperationException("Method is no longer supported.");
+ }
+
+ /**
+ * Creates a new table from parameters.
+ *
+ * @param parameters Create table parameters.
+ * @return Future representing pending completion of the operation.
+ * @see TableAlreadyExistsException
+ */
+ @Deprecated(forRemoval = true)
+ public CompletableFuture<Table> createTableAsync(CreateTableParams parameters) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
}
try {
- return createTableAsyncInternal(name, zoneName, tableInitChange);
+ String tableName = parameters.tableName();
+ String zoneName = Objects.requireNonNullElse(parameters.zone(), DEFAULT_ZONE_NAME);
+
+ // Copied from DdlCommandHandler
+ Consumer<TableChange> tblChanger = tableChange -> {
+ tableChange.changeColumns(columnsChange -> {
+ for (var col : parameters.columns()) {
+ columnsChange.create(col.name(), columnChange -> CatalogDescriptorUtils.convertColumnDefinition(col, columnChange));
+ }
+ });
+
+ var colocationKeys = parameters.colocationColumns();
+
+ if (nullOrEmpty(colocationKeys)) {
+ colocationKeys = parameters.primaryKeyColumns();
+ }
+
+ var colocationKeys0 = colocationKeys;
+
+ tableChange.changePrimaryKey(pkChange -> pkChange.changeColumns(parameters.primaryKeyColumns().toArray(String[]::new))
+ .changeColocationColumns(colocationKeys0.toArray(String[]::new)));
+ };
+
+ return catalogManager.createTable(parameters)
+ .thenApply(ignore -> catalogManager.table(tableName, Long.MAX_VALUE).id())
+ .thenCompose(tableId -> createTableAsyncInternal(tableId, tableName, zoneName, tblChanger));
} finally {
busyLock.leaveBusy();
}
}
- /** See {@link #createTableAsync(String, String, Consumer)} for details. */
+ /** See {@link #createTableAsync(CreateTableParams)} for details. */
+ @Deprecated(forRemoval = true)
private CompletableFuture<Table> createTableAsyncInternal(
+ int tableId,
String name,
String zoneName,
Consumer<TableChange> tableInitChange
@@ -1536,6 +1595,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
try {
changeTablesConfigurationOnTableCreate(
+ tableId,
name,
zoneId,
tableInitChange,
@@ -1571,12 +1631,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
/**
* Creates a new table in {@link TablesConfiguration}.
*
+ * @param tableId Table id.
* @param name Table name.
* @param zoneId Distribution zone id.
* @param tableInitChange Table changer.
* @param tblFut Future representing pending completion of the table creation.
*/
private void changeTablesConfigurationOnTableCreate(
+ int tableId,
String name,
int zoneId,
Consumer<TableChange> tableInitChange,
@@ -1597,8 +1659,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
var extConfCh = ((ExtendedTableChange) tableChange);
- int tableId = tablesChange.globalIdCounter() + 1;
-
extConfCh.changeId(tableId);
tablesChange.changeGlobalIdCounter(tableId);
@@ -1641,12 +1701,46 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* </ul>
* @see TableNotFoundException
*/
+ @Deprecated(forRemoval = true)
public CompletableFuture<Void> alterTableAsync(String name, Function<TableChange, Boolean> tableChange) {
+ throw new UnsupportedOperationException("Method is no longer supported.");
+ }
+
+ /**
+ * Alters a cluster table and adds columns to the table.
+ *
+ * @param params Create column params.
+ * @return Future representing pending completion of the operation.
+ * @see TableNotFoundException
+ */
+ @Deprecated
+ public CompletableFuture<Void> alterTableAddColumnAsync(AlterTableAddColumnParams params) {
+ if (!busyLock.enterBusy()) {
+ throw new IgniteException(new NodeStoppingException());
+ }
+ try {
+ return catalogManager.addColumn(params)
+ .thenCompose(ignore -> addColumnInternal(params.tableName(), params.columns()));
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ /**
+ * Alters a cluster table and drops columns from the table.
+ *
+ * @param params Drop column params.
+ * @return Future representing pending completion of the operation.
+ * @see TableNotFoundException
+ */
+ @Deprecated
+ public CompletableFuture<Void> alterTableDropColumnAsync(AlterTableDropColumnParams params) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
}
try {
- return alterTableAsyncInternal(name, tableChange);
+ return catalogManager.dropColumn(params)
+ .thenCompose(ignore -> dropColumnInternal(params.tableName(), params.columns()));
} finally {
busyLock.leaveBusy();
}
@@ -1736,18 +1830,33 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* </ul>
* @see TableNotFoundException
*/
+ @Deprecated(forRemoval = true)
public CompletableFuture<Void> dropTableAsync(String name) {
+ throw new UnsupportedOperationException("Method is no longer supported.");
+ }
+
+ /**
+ * Drops a table with the name specified. If appropriate table does not be found, a future will be completed with
+ * {@link TableNotFoundException}.
+ *
+ * @param params Drop table parameters.
+ * @return Future representing pending completion of the operation.
+ * @see TableNotFoundException
+ */
+ @Deprecated
+ public CompletableFuture<Void> dropTableAsync(DropTableParams params) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
}
try {
- return dropTableAsyncInternal(name);
+ return catalogManager.dropTable(params)
+ .thenCompose(ignore -> dropTableAsyncInternal(params.tableName()));
} finally {
busyLock.leaveBusy();
}
}
- /** See {@link #dropTableAsync(String)} for details. */
+ /** See {@link #dropTableAsync(DropTableParams)} for details. */
private CompletableFuture<Void> dropTableAsyncInternal(String name) {
return tableAsyncInternal(name).thenCompose(tbl -> {
// In case of drop it's an optimization that allows not to fire drop-change-closure if there's no such
@@ -2718,4 +2827,79 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
private CatalogZoneDescriptor getZoneDescriptor(int id) {
return toZoneDescriptor(getZoneById(zonesConfig, id).value());
}
+
+ // Copied from DdlCommandHandler
+ @Deprecated(forRemoval = true)
+ private CompletableFuture<Void> addColumnInternal(String fullName, List<ColumnParams> columnParams) {
+ return alterTableAsyncInternal(
+ fullName,
+ chng -> {
+ if (columnParams.isEmpty()) {
+ return false;
+ }
+
+ chng.changeColumns(cols -> {
+ Set<String> colNamesToOrders = new HashSet<>(cols.namedListKeys());
+
+ columnParams.stream()
+ .filter(k -> colNamesToOrders.contains(k.name()))
+ .findAny()
+ .ifPresent(c -> {
+ throw new ColumnAlreadyExistsException(c.name());
+ });
+
+ for (ColumnParams col : columnParams) {
+ cols.create(col.name(), colChg -> CatalogDescriptorUtils.convertColumnDefinition(col, colChg));
+ }
+ });
+
+ return true;
+ }
+ );
+ }
+
+ // Copied from DdlCommandHandler
+ // TODO: IGNITE-19082 Drop unused temporary method.
+ @Deprecated(forRemoval = true)
+ private CompletableFuture<Void> dropColumnInternal(String tableName, Set<String> colNames) {
+ AtomicBoolean ret = new AtomicBoolean(true);
+
+ return alterTableAsyncInternal(
+ tableName,
+ chng -> {
+ chng.changeColumns(cols -> {
+ ret.set(true); // Reset state if closure have been restarted.
+
+ PrimaryKeyView priKey = chng.primaryKey();
+
+ Set<String> colNamesToOrders = new HashSet<>(cols.namedListKeys());
+
+ Set<String> colNames0 = new HashSet<>();
+
+ Set<String> primaryCols = Set.of(priKey.columns());
+
+ // Catalog verification passe, so we can omit validation here.
+ // reportIndexedColumns(tableName, colNames, primaryCols);
+
+ for (String colName : colNames) {
+ if (!colNamesToOrders.contains(colName)) {
+ ret.set(false);
+
+ throw new ColumnNotFoundException(DEFAULT_SCHEMA_NAME, tableName, colName);
+ } else {
+ colNames0.add(colName);
+ }
+
+ if (primaryCols.contains(colName)) {
+ throw new SqlException(DROP_IDX_COLUMN_CONSTRAINT_ERR, IgniteStringFormatter
+ .format("Can`t delete column, belongs to primary key: [name={}]", colName));
+ }
+ }
+
+ colNames0.forEach(cols::delete);
+ });
+
+ return ret.get();
+ });
+ }
}
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index 898358beeb..883c0bf771 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -37,6 +37,8 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
@@ -60,11 +62,13 @@ import java.util.concurrent.Flow.Subscription;
import java.util.concurrent.Phaser;
import java.util.concurrent.TimeUnit;
import java.util.function.Consumer;
-import java.util.function.Function;
import java.util.function.LongFunction;
import org.apache.ignite.configuration.NamedListView;
import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.baseline.BaselineManager;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
+import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
@@ -77,8 +81,6 @@ import org.apache.ignite.internal.configuration.testframework.InjectRevisionList
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.hlc.HybridClockImpl;
-import org.apache.ignite.internal.logger.IgniteLogger;
-import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.metastorage.MetaStorageManager;
import org.apache.ignite.internal.metastorage.dsl.Operation;
import org.apache.ignite.internal.raft.Loza;
@@ -93,10 +95,10 @@ import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.SchemaUtils;
import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
-import org.apache.ignite.internal.schema.configuration.TableChange;
import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.testutils.SchemaConfigurationConverter;
+import org.apache.ignite.internal.schema.testutils.SchemaToCatalogParamsConverter;
import org.apache.ignite.internal.schema.testutils.builder.SchemaBuilders;
import org.apache.ignite.internal.schema.testutils.definition.ColumnType;
import org.apache.ignite.internal.schema.testutils.definition.TableDefinition;
@@ -143,8 +145,6 @@ import org.mockito.quality.Strictness;
@ExtendWith({MockitoExtension.class, ConfigurationExtension.class})
@MockitoSettings(strictness = Strictness.LENIENT)
public class TableManagerTest extends IgniteAbstractTest {
- private static final IgniteLogger LOG = Loggers.forClass(TableManagerTest.class);
-
/** The name of the table which is preconfigured. */
private static final String PRECONFIGURED_TABLE_NAME = "T1";
@@ -388,7 +388,7 @@ public class TableManagerTest extends IgniteAbstractTest {
TableManager tableManager = tblManagerFut.join();
- await(tableManager.dropTableAsync(DYNAMIC_TABLE_FOR_DROP_NAME));
+ await(tableManager.dropTableAsync(DropTableParams.builder().schemaName("PUBLIC").tableName(DYNAMIC_TABLE_FOR_DROP_NAME).build()));
verify(mvTableStorage).destroy();
verify(txStateTableStorage).destroy();
@@ -413,33 +413,33 @@ public class TableManagerTest extends IgniteAbstractTest {
createDistributionZone();
- Consumer<TableChange> createTableChange = (TableChange change) ->
- SchemaConfigurationConverter.convert(SchemaBuilders.tableBuilder("PUBLIC", DYNAMIC_TABLE_FOR_DROP_NAME).columns(
- SchemaBuilders.column("key", ColumnType.INT64).build(),
- SchemaBuilders.column("val", ColumnType.INT64).asNullable(true).build()
- ).withPrimaryKey("key").build(), change);
-
- Function<TableChange, Boolean> addColumnChange = (TableChange change) -> {
- change.changeColumns(cols -> {
- int colIdx = change.columns().namedListKeys().stream().mapToInt(Integer::parseInt).max().getAsInt() + 1;
-
- cols.create(String.valueOf(colIdx),
- colChg -> SchemaConfigurationConverter.convert(SchemaBuilders.column("name", ColumnType.string()).build(),
- colChg));
-
- });
+ TableDefinition tableDef = SchemaBuilders.tableBuilder("PUBLIC", DYNAMIC_TABLE_FOR_DROP_NAME)
+ .columns(
+ SchemaBuilders.column("key", ColumnType.INT64).build(),
+ SchemaBuilders.column("val", ColumnType.INT64).asNullable(true).build()
+ )
+ .withPrimaryKey("key")
+ .build();
- return true;
- };
+ AlterTableAddColumnParams addColumnParams = AlterTableAddColumnParams.builder()
+ .schemaName("PUBLIC")
+ .tableName(DYNAMIC_TABLE_FOR_DROP_NAME)
+ .columns(List.of(
+ SchemaToCatalogParamsConverter.convert(SchemaBuilders.column("name", ColumnType.string()).build())
+ ))
+ .build();
TableManager igniteTables = tableManager;
assertThrows(IgniteException.class,
- () -> igniteTables.createTableAsync(DYNAMIC_TABLE_FOR_DROP_NAME, ZONE_NAME, createTableChange));
+ () -> igniteTables.createTableAsync(SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, tableDef)));
- assertThrows(IgniteException.class, () -> igniteTables.alterTableAsync(DYNAMIC_TABLE_FOR_DROP_NAME, addColumnChange));
+ assertThrows(IgniteException.class, () -> igniteTables.alterTableAddColumnAsync(addColumnParams));
- assertThrows(IgniteException.class, () -> igniteTables.dropTableAsync(DYNAMIC_TABLE_FOR_DROP_NAME));
+ assertThrows(IgniteException.class, () -> igniteTables.dropTableAsync(DropTableParams.builder()
+ .schemaName(tableDef.schemaName())
+ .tableName(tableDef.name())
+ .build()));
assertThrows(IgniteException.class, () -> igniteTables.tables());
assertThrows(IgniteException.class, () -> igniteTables.tablesAsync());
@@ -632,8 +632,9 @@ public class TableManagerTest extends IgniteAbstractTest {
assertNotNull(table);
assertThrows(RuntimeException.class,
- () -> await(tblManagerFut.join().createTableAsync(DYNAMIC_TABLE_NAME, ZONE_NAME,
- tblCh -> SchemaConfigurationConverter.convert(scmTbl, tblCh))));
+ () -> await(tblManagerFut.join().createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, scmTbl)
+ )));
assertSame(table, tblManagerFut.join().table(scmTbl.name()));
}
@@ -822,8 +823,8 @@ public class TableManagerTest extends IgniteAbstractTest {
createDistributionZone();
- CompletableFuture<Table> tbl2Fut = tableManager.createTableAsync(tableDefinition.name(), ZONE_NAME,
- tblCh -> SchemaConfigurationConverter.convert(tableDefinition, tblCh)
+ CompletableFuture<Table> tbl2Fut = tableManager.createTableAsync(
+ SchemaToCatalogParamsConverter.toCreateTable(ZONE_NAME, tableDefinition)
);
assertTrue(createTblLatch.await(10, TimeUnit.SECONDS));
@@ -857,6 +858,14 @@ public class TableManagerTest extends IgniteAbstractTest {
when(vaultManager.get(any(ByteArray.class))).thenReturn(completedFuture(null));
when(vaultManager.put(any(ByteArray.class), any(byte[].class))).thenReturn(completedFuture(null));
+ //TODO IGNITE-19082 drop mocked catalog manager.
+ CatalogManager catalogManager = mock(CatalogManager.class);
+ CatalogTableDescriptor tableDescriptor = mock(CatalogTableDescriptor.class);
+ when(catalogManager.createTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.dropTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.table(anyString(), anyLong())).thenReturn(tableDescriptor);
+ when(tableDescriptor.id()).thenReturn(1);
+
TableManager tableManager = new TableManager(
"test",
revisionUpdater,
@@ -874,6 +883,7 @@ public class TableManagerTest extends IgniteAbstractTest {
dsm = createDataStorageManager(configRegistry, workDir, storageEngineConfig),
workDir,
msm,
+ catalogManager,
sm = new SchemaManager(revisionUpdater, tblsCfg, msm),
budgetView -> new LocalLogStorageFactory(),
new HybridClockImpl(),
[ignite-3] 05/19: Minor.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 113bc82748868c327ad31bb9573e8af46cfad3b5
Author: amashenkov <an...@gmail.com>
AuthorDate: Wed Jul 19 12:25:52 2023 +0300
Minor.
---
.../java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index 24f413484c..02e9ca4209 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -154,7 +154,7 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
// TODO: IGNITE-19082 Move default schema objects initialization to cluster init procedure.
CatalogSchemaDescriptor schemaPublic = new CatalogSchemaDescriptor(
- objectIdGen++,
+ 0,
DEFAULT_SCHEMA_NAME,
new CatalogTableDescriptor[0],
new CatalogIndexDescriptor[0]
[ignite-3] 07/19: Switch TableManager to Catalog events.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 8221157b789d158ba3fa52a45bc55b7f2040857a
Author: amashenkov <an...@gmail.com>
AuthorDate: Wed Jul 12 18:04:37 2023 +0300
Switch TableManager to Catalog events.
---
.../internal/catalog/CatalogManagerImpl.java | 11 +++++
.../ignite/internal/catalog/CatalogService.java | 2 +
.../apache/ignite/internal/index/IndexManager.java | 2 +-
.../sql/engine/rule/logical/ExposeIndexRule.java | 9 ++--
.../engine/rule/logical/LogicalOrToUnionRule.java | 12 +++---
.../sql/engine/schema/IgniteSchemaTable.java | 6 ---
.../internal/sql/engine/schema/IgniteTable.java | 12 ------
.../internal/table/distributed/TableManager.java | 49 +++++++++-------------
.../table/distributed/TableManagerTest.java | 24 +++++------
9 files changed, 56 insertions(+), 71 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index 02e9ca4209..d441478fe6 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -255,6 +255,17 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
return catalogAt(timestamp).zone(zoneId);
}
+ @Override
+ public CatalogZoneDescriptor zone(int zoneId, int version) {
+ Catalog catalog = catalog(version);
+
+ if (catalog == null) {
+ return null;
+ }
+
+ return catalog.zone(zoneId);
+ }
+
@Override
public @Nullable CatalogSchemaDescriptor activeSchema(long timestamp) {
return catalogAt(timestamp).schema(DEFAULT_SCHEMA_NAME);
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
index c1d30f1b4c..44ab7ed8de 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
@@ -65,6 +65,8 @@ public interface CatalogService {
CatalogZoneDescriptor zone(int zoneId, long timestamp);
+ CatalogZoneDescriptor zone(int zoneId, int version);
+
CatalogSchemaDescriptor activeSchema(long timestamp);
CatalogSchemaDescriptor activeSchema(@Nullable String schemaName, long timestamp);
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index d14dc9dfd5..ff34697c24 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -528,7 +528,7 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
});
- return allOf(createIndexFuture, fireEventFuture);
+ return createIndexFuture;
}
/**
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
index ffd52389dd..d9078b4d4c 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
@@ -34,6 +34,7 @@ import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
+import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaTable;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.immutables.value.Value;
@@ -51,7 +52,7 @@ public class ExposeIndexRule extends RelRule<ExposeIndexRule.Config> {
private static boolean preMatch(IgniteLogicalTableScan scan) {
// has indexes to expose
- return !scan.getTable().unwrap(IgniteTable.class).indexes().isEmpty();
+ return !scan.getTable().unwrap(IgniteSchemaTable.class).getIndexes().isEmpty();
}
/** {@inheritDoc} */
@@ -61,13 +62,13 @@ public class ExposeIndexRule extends RelRule<ExposeIndexRule.Config> {
RelOptCluster cluster = scan.getCluster();
RelOptTable optTable = scan.getTable();
- IgniteTable igniteTable = optTable.unwrap(IgniteTable.class);
+ IgniteSchemaTable igniteTable = optTable.unwrap(IgniteSchemaTable.class);
List<RexNode> proj = scan.projects();
RexNode condition = scan.condition();
ImmutableBitSet requiredCols = scan.requiredColumns();
- List<IgniteLogicalIndexScan> indexes = igniteTable.indexes().keySet().stream()
- .map(idxName -> igniteTable.toRel(cluster, optTable, idxName, proj, condition, requiredCols))
+ List<IgniteLogicalIndexScan> indexes = igniteTable.getIndexes().values().stream()
+ .map(idx -> idx.toRel(cluster, optTable, proj, condition, requiredCols))
.filter(idx -> filter(igniteTable, idx.indexName(), idx.searchBounds()))
.collect(Collectors.toList());
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
index 2c58e626fc..eb2bb7d442 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
@@ -37,9 +37,8 @@ import org.apache.calcite.sql.SqlKind;
import org.apache.calcite.tools.RelBuilder;
import org.apache.calcite.util.mapping.Mappings;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
-import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
+import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaTable;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.Commons;
import org.apache.ignite.internal.util.CollectionUtils;
@@ -132,15 +131,14 @@ public class LogicalOrToUnionRule extends RelRule<LogicalOrToUnionRule.Config> {
private boolean idxCollationCheck(RelOptRuleCall call, List<RexNode> operands) {
final IgniteLogicalTableScan scan = call.rel(0);
- IgniteTable tbl = scan.getTable().unwrap(IgniteTable.class);
+ IgniteSchemaTable tbl = scan.getTable().unwrap(IgniteSchemaTable.class);
IgniteTypeFactory typeFactory = Commons.typeFactory(scan.getCluster());
int fieldCnt = tbl.getRowType(typeFactory).getFieldCount();
BitSet idxsFirstFields = new BitSet(fieldCnt);
- for (IgniteIndex idx : tbl.indexes().values()) {
- List<RelFieldCollation> fieldCollations = TraitUtils.createCollation(idx.columns(), idx.collations(), tbl.descriptor())
- .getFieldCollations();
+ for (IgniteSchemaIndex idx : tbl.getIndexes().values()) {
+ List<RelFieldCollation> fieldCollations = idx.collation().getFieldCollations();
if (!CollectionUtils.nullOrEmpty(fieldCollations)) {
idxsFirstFields.set(fieldCollations.get(0).getFieldIndex());
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
index 36d7aea231..4181054e0d 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
@@ -119,12 +119,6 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
return toRel(cluster, relOptTbl, hints, null, null, null);
}
- /** {@inheritDoc} */
- @Override
- public IgniteLogicalIndexScan toRel(RelOptCluster cluster, RelOptTable relOptTbl, String idxName) {
- return toRel(cluster, relOptTbl, idxName, null, null, null);
- }
-
/** {@inheritDoc} */
@Override
public IgniteLogicalTableScan toRel(
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTable.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTable.java
index cb39f4dad2..2193789be0 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTable.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTable.java
@@ -106,18 +106,6 @@ public interface IgniteTable extends TranslatableTable, Wrapper {
return toRel(cluster, relOptTbl, hints, null, null, null);
}
- /**
- * Converts table into relational expression.
- *
- * @param cluster Custer.
- * @param relOptTbl Table.
- * @param idxName Index name.
- * @return Table relational expression.
- */
- default IgniteLogicalIndexScan toRel(RelOptCluster cluster, RelOptTable relOptTbl, String idxName) {
- return toRel(cluster, relOptTbl, idxName, null, null, null);
- }
-
/**
* Converts table into table scan relational expression.
*/
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 41f3a4d886..a3fd1e03d1 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -87,8 +87,6 @@ import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.apache.ignite.configuration.ConfigurationChangeException;
import org.apache.ignite.configuration.ConfigurationProperty;
-import org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
-import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.affinity.Assignment;
import org.apache.ignite.internal.baseline.BaselineManager;
@@ -101,6 +99,10 @@ import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CatalogEventParameters;
+import org.apache.ignite.internal.catalog.events.CreateTableEventParameters;
+import org.apache.ignite.internal.catalog.events.DropTableEventParameters;
import org.apache.ignite.internal.causality.CompletionListener;
import org.apache.ignite.internal.causality.IncrementalVersionedValue;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
@@ -529,24 +531,11 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
metaStorageMgr.registerPrefixWatch(ByteArray.fromString(STABLE_ASSIGNMENTS_PREFIX), stableAssignmentsRebalanceListener);
metaStorageMgr.registerPrefixWatch(ByteArray.fromString(ASSIGNMENTS_SWITCH_REDUCE_PREFIX), assignmentsSwitchRebalanceListener);
- tablesCfg.tables().listenElements(new ConfigurationNamedListListener<>() {
- @Override
- public CompletableFuture<?> onCreate(ConfigurationNotificationEvent<TableView> ctx) {
- return onTableCreate(ctx);
- }
-
- @Override
- public CompletableFuture<?> onRename(ConfigurationNotificationEvent<TableView> ctx) {
- // TODO: IGNITE-15485 Support table rename operation.
+ catalogManager.listen(CatalogEvent.TABLE_CREATE,
+ (parameters, exception) -> onTableCreate((CreateTableEventParameters) parameters).thenApply(ignore -> false));
- return completedFuture(null);
- }
-
- @Override
- public CompletableFuture<?> onDelete(ConfigurationNotificationEvent<TableView> ctx) {
- return onTableDelete(ctx);
- }
- });
+ catalogManager.listen(CatalogEvent.TABLE_CREATE,
+ (parameters, exception) -> onTableDelete((DropTableEventParameters) parameters).thenApply(ignore -> false));
schemaManager.listen(SchemaEvent.CREATE, new EventListener<>() {
@Override
@@ -603,10 +592,10 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param ctx Table configuration context.
* @return A future.
*/
- private CompletableFuture<?> onTableCreate(ConfigurationNotificationEvent<TableView> ctx) {
+ private CompletableFuture<?> onTableCreate(CreateTableEventParameters ctx) {
if (!busyLock.enterBusy()) {
fireEvent(TableEvent.CREATE,
- new TableEventParameters(ctx.storageRevision(), ctx.newValue().id()),
+ new TableEventParameters(ctx.causalityToken(), ctx.tableDescriptor().id()),
new NodeStoppingException()
);
@@ -614,8 +603,10 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- CatalogTableDescriptor tableDescriptor = toTableDescriptor(ctx.newValue());
- CatalogZoneDescriptor zoneDescriptor = getZoneDescriptor(tableDescriptor.zoneId());
+ CatalogTableDescriptor tableDescriptor = ctx.tableDescriptor();
+ CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), ctx.catalogVersion());
+
+ assert zoneDescriptor != null;
List<Set<Assignment>> assignments;
@@ -637,7 +628,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
assert !assignments.isEmpty() : "Couldn't create the table with empty assignments.";
CompletableFuture<?> createTableFut = createTableLocally(
- ctx.storageRevision(),
+ ctx.causalityToken(),
tableDescriptor,
zoneDescriptor,
assignments
@@ -688,11 +679,11 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param ctx Table configuration context.
* @return A future.
*/
- private CompletableFuture<?> onTableDelete(ConfigurationNotificationEvent<TableView> ctx) {
+ private CompletableFuture<?> onTableDelete(DropTableEventParameters ctx) {
if (!busyLock.enterBusy()) {
fireEvent(
TableEvent.DROP,
- new TableEventParameters(ctx.storageRevision(), ctx.oldValue().id()),
+ new TableEventParameters(ctx.causalityToken(), ctx.tableId()),
new NodeStoppingException()
);
@@ -700,10 +691,10 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- CatalogTableDescriptor tableDescriptor = toTableDescriptor(ctx.oldValue());
- CatalogZoneDescriptor zoneDescriptor = getZoneDescriptor(tableDescriptor.zoneId());
+ CatalogTableDescriptor tableDescriptor = catalogManager.table(ctx.tableId(), ctx.catalogVersion() - 1);
+ CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), ctx.catalogVersion() - 1);
- dropTableLocally(ctx.storageRevision(), tableDescriptor, zoneDescriptor);
+ dropTableLocally(ctx.causalityToken(), tableDescriptor, zoneDescriptor);
} finally {
busyLock.leaveBusy();
}
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index 83b5e6f5ff..22aa333a72 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -23,6 +23,7 @@ import static org.apache.ignite.internal.distributionzones.DistributionZoneManag
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.getZoneById;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
+import static org.apache.ignite.internal.testframework.IgniteTestUtils.cause;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willSucceedFast;
import static org.hamcrest.MatcherAssert.assertThat;
@@ -67,6 +68,8 @@ import org.apache.ignite.configuration.NamedListView;
import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.CatalogServiceImpl;
+import org.apache.ignite.internal.catalog.ClockWaiter;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
@@ -74,6 +77,7 @@ import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
+import org.apache.ignite.internal.catalog.storage.UpdateLogImpl;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologySnapshot;
import org.apache.ignite.internal.configuration.ConfigurationRegistry;
@@ -81,11 +85,13 @@ import org.apache.ignite.internal.configuration.notifications.ConfigurationStora
import org.apache.ignite.internal.configuration.testframework.ConfigurationExtension;
import org.apache.ignite.internal.configuration.testframework.InjectConfiguration;
import org.apache.ignite.internal.configuration.testframework.InjectRevisionListenerHolder;
+import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.hlc.HybridClockImpl;
import org.apache.ignite.internal.metastorage.MetaStorageManager;
import org.apache.ignite.internal.metastorage.dsl.Operation;
+import org.apache.ignite.internal.metastorage.impl.StandaloneMetaStorageManager;
import org.apache.ignite.internal.raft.Loza;
import org.apache.ignite.internal.raft.Peer;
import org.apache.ignite.internal.raft.client.TopologyAwareRaftGroupService;
@@ -862,18 +868,12 @@ public class TableManagerTest extends IgniteAbstractTest {
when(vaultManager.put(any(ByteArray.class), any(byte[].class))).thenReturn(completedFuture(null));
//TODO IGNITE-19082 drop mocked catalog manager.
- CatalogManager catalogManager = mock(CatalogManager.class);
- CatalogTableDescriptor tableDescriptor = mock(CatalogTableDescriptor.class);
- CatalogIndexDescriptor indexDescriptor = mock(CatalogIndexDescriptor.class);
- when(tableDescriptor.id()).thenReturn(1);
- when(indexDescriptor.id()).thenReturn(1);
- when(catalogManager.createTable(any())).thenReturn(completedFuture(null));
- when(catalogManager.createIndex(any(CreateHashIndexParams.class))).thenReturn(completedFuture(null));
- when(catalogManager.createIndex(any(CreateSortedIndexParams.class))).thenReturn(completedFuture(null));
- when(catalogManager.dropTable(any())).thenReturn(completedFuture(null));
- when(catalogManager.dropIndex(any())).thenReturn(completedFuture(null));
- when(catalogManager.table(anyString(), anyLong())).thenReturn(tableDescriptor);
- when(catalogManager.index(anyString(), anyLong())).thenReturn(indexDescriptor);
+
+ CatalogManager catalogManager = new CatalogServiceImpl(
+ new UpdateLogImpl(StandaloneMetaStorageManager.create(vaultManager)),
+ new ClockWaiter("note", new HybridClockImpl()));
+
+ catalogManager.start();
TableManager tableManager = new TableManager(
"test",
[ignite-3] 18/19: wip.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 8680b28248839d728430baf614ca2f55213c37ff
Author: amashenkov <an...@gmail.com>
AuthorDate: Tue Jul 25 16:14:28 2023 +0300
wip.
---
.../sql/engine/exec/ExecutionServiceImpl.java | 3 ++-
.../sql/engine/exec/ddl/DdlCommandHandler.java | 25 ++++++++++++++++------
.../engine/rule/logical/LogicalOrToUnionRule.java | 4 ++--
.../sql/engine/trait/DistributionFunction.java | 4 +++-
.../sql/engine/util/HashFunctionFactoryImpl.java | 22 ++++++++++++++++++-
.../internal/table/distributed/TableManager.java | 18 ----------------
6 files changed, 46 insertions(+), 30 deletions(-)
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
index 3cf9154624..3472b6a133 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
@@ -79,6 +79,7 @@ import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableModify;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableScan;
import org.apache.ignite.internal.sql.engine.rel.SourceAwareIgniteRel;
+import org.apache.ignite.internal.sql.engine.schema.IgniteCatalogSchema;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManager;
import org.apache.ignite.internal.sql.engine.util.BaseQueryContext;
@@ -173,7 +174,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
dependencyResolver,
(ctx, deps) -> new LogicalRelImplementor<>(
ctx,
- new HashFunctionFactoryImpl<>(sqlSchemaManager, handler),
+ new HashFunctionFactoryImpl<>(ctx.getRootSchema().unwrap(IgniteCatalogSchema.class), handler),
mailboxRegistry,
exchangeSrvc,
deps),
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index 9c9b9622bc..ce5daac344 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -26,6 +26,9 @@ import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
import java.util.function.BiFunction;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.AbstractIndexCommandParams;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.index.IndexManager;
@@ -215,13 +218,13 @@ public class DdlCommandHandler {
/** Handles create table command. */
private CompletableFuture<Boolean> handleCreateTable(CreateTableCommand cmd) {
- return tableManager.createTableAsync(DdlToCatalogCommandConverter.convert(cmd))
+ return catalogManager.createTable(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableAlreadyExistsException.class));
}
/** Handles drop table command. */
private CompletableFuture<Boolean> handleDropTable(DropTableCommand cmd) {
- return tableManager.dropTableAsync(DdlToCatalogCommandConverter.convert(cmd))
+ return catalogManager.dropTable(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
@@ -231,7 +234,7 @@ public class DdlCommandHandler {
return completedFuture(Boolean.FALSE);
}
- return tableManager.alterTableAddColumnAsync(DdlToCatalogCommandConverter.convert(cmd))
+ return catalogManager.addColumn(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
@@ -241,7 +244,7 @@ public class DdlCommandHandler {
return completedFuture(Boolean.FALSE);
}
- return tableManager.alterTableDropColumnAsync(DdlToCatalogCommandConverter.convert(cmd))
+ return catalogManager.dropColumn(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
@@ -269,13 +272,21 @@ public class DdlCommandHandler {
/** Handles create index command. */
private CompletableFuture<Boolean> handleCreateIndex(CreateIndexCommand cmd) {
- return indexManager.createIndexAsync(DdlToCatalogCommandConverter.convert(cmd))
- .handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
+ AbstractIndexCommandParams params = DdlToCatalogCommandConverter.convert(cmd);
+
+ if (params instanceof CreateSortedIndexParams) {
+ return catalogManager.createIndex((CreateSortedIndexParams) params)
+ .handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
+ }
+ else {
+ return catalogManager.createIndex((CreateHashIndexParams) params)
+ .handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
+ }
}
/** Handles drop index command. */
private CompletableFuture<Boolean> handleDropIndex(DropIndexCommand cmd) {
- return indexManager.dropIndexAsync(DdlToCatalogCommandConverter.convert(cmd))
+ return catalogManager.dropIndex(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifNotExists(), IndexNotFoundException.class));
}
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
index 0612c7612e..3676e75b0e 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
@@ -38,7 +38,7 @@ import org.apache.calcite.tools.RelBuilder;
import org.apache.calcite.util.mapping.Mappings;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaTable;
+import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.Commons;
import org.apache.ignite.internal.util.CollectionUtils;
@@ -131,7 +131,7 @@ public class LogicalOrToUnionRule extends RelRule<LogicalOrToUnionRule.Config> {
private boolean idxCollationCheck(RelOptRuleCall call, List<RexNode> operands) {
final IgniteLogicalTableScan scan = call.rel(0);
- IgniteSchemaTable tbl = scan.getTable().unwrap(IgniteSchemaTable.class);
+ IgniteTable tbl = scan.getTable().unwrap(IgniteTable.class);
IgniteTypeFactory typeFactory = Commons.typeFactory(scan.getCluster());
int fieldCnt = tbl.getRowType(typeFactory).getFieldCount();
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/DistributionFunction.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/DistributionFunction.java
index ed5e8584a2..d65057b8fc 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/DistributionFunction.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/DistributionFunction.java
@@ -222,7 +222,9 @@ public abstract class DistributionFunction {
/** {@inheritDoc} */
@Override
public <RowT> Destination<RowT> destination(HashFunctionFactory<RowT> hashFuncFactory, ColocationGroup m, ImmutableIntList k) {
- assert m != null && !nullOrEmpty(m.assignments()) && !k.isEmpty();
+ if (! (m != null && !nullOrEmpty(m.assignments()) && !k.isEmpty())) {
+ throw new AssertionError();
+ }
List<List<String>> assignments = Commons.transform(m.assignments(), v -> Commons.transform(v, NodeWithTerm::name));
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
index 4ad5484257..6f583b43b6 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
@@ -26,6 +26,8 @@ import org.apache.ignite.internal.schema.NativeType;
import org.apache.ignite.internal.schema.NativeTypeSpec;
import org.apache.ignite.internal.sql.engine.exec.RowHandler;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
+import org.apache.ignite.internal.sql.engine.schema.IgniteCatalogSchema;
+import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManager;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.util.ColocationUtils;
@@ -36,11 +38,20 @@ import org.apache.ignite.internal.util.HashCalculator;
*/
public class HashFunctionFactoryImpl<T> implements HashFunctionFactory<T> {
private final SqlSchemaManager sqlSchemaManager;
+ private final IgniteCatalogSchema sqlSchema;
private final RowHandler<T> rowHandler;
+ @Deprecated(forRemoval = true)
public HashFunctionFactoryImpl(SqlSchemaManager sqlSchemaManager, RowHandler<T> rowHandler) {
this.sqlSchemaManager = sqlSchemaManager;
this.rowHandler = rowHandler;
+ this.sqlSchema = null;
+ }
+
+ public HashFunctionFactoryImpl(IgniteCatalogSchema sqlSchema, RowHandler<T> rowHandler) {
+ this.sqlSchemaManager = null;
+ this.rowHandler = rowHandler;
+ this.sqlSchema = sqlSchema;
}
/** {@inheritDoc} */
@@ -48,7 +59,16 @@ public class HashFunctionFactoryImpl<T> implements HashFunctionFactory<T> {
public RowHashFunction<T> create(int[] fields, int tableId) {
int fieldCnt = fields.length;
NativeType[] fieldTypes = new NativeType[fieldCnt];
- TableDescriptor tblDesc = sqlSchemaManager.tableById(tableId).descriptor();
+
+ //TODO: optimize this
+ TableDescriptor tblDesc = sqlSchema.getTableNames().stream()
+ .map(sqlSchema::getTable)
+ .map(IgniteTable.class::cast)
+ .filter(t -> t.id() == tableId)
+ .map(IgniteTable::descriptor)
+ .findFirst()
+ .get();
+
ImmutableIntList colocationColumns = tblDesc.distribution().getKeys();
assert colocationColumns.size() == fieldCnt : "fieldsCount=" + fieldCnt + ", colocationColumns=" + colocationColumns;
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index afbc7093be..b923703efe 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -1491,24 +1491,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
}
- /**
- * Alters a cluster table and drops columns from the table.
- *
- * @param params Drop column params.
- * @return Future representing pending completion of the operation.
- * @see TableNotFoundException
- */
- public CompletableFuture<Void> alterTableDropColumnAsync(AlterTableDropColumnParams params) {
- if (!busyLock.enterBusy()) {
- throw new IgniteException(new NodeStoppingException());
- }
- try {
- return catalogManager.dropColumn(params);
- } finally {
- busyLock.leaveBusy();
- }
- }
-
/**
* Drops a table with the name specified. If appropriate table does not be found, a future will be completed with
* {@link TableNotFoundException}.
[ignite-3] 03/19: Mirror Index changes from Config to Catalog
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit d4573598cc3c734cc8a7ce9fa59ae185eed8e5dd
Author: amashenkov <an...@gmail.com>
AuthorDate: Tue Jul 11 19:30:31 2023 +0300
Mirror Index changes from Config to Catalog
---
.../internal/catalog/commands/CatalogUtils.java | 1 -
.../apache/ignite/internal/index/IndexManager.java | 139 ++++++++++++++++++++-
.../ignite/internal/index/IndexManagerTest.java | 90 +++++++++----
.../ignite/internal/index/ItIndexManagerTest.java | 23 ++--
.../runner/app/ItIgniteNodeRestartTest.java | 8 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 2 +-
.../sql/engine/exec/ddl/DdlCommandHandler.java | 2 +-
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 37 +++---
.../sql/engine/exec/MockedStructuresTest.java | 17 ++-
.../internal/table/distributed/TableManager.java | 8 +-
.../table/distributed/TableManagerTest.java | 11 +-
11 files changed, 271 insertions(+), 67 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index d157cbaba2..7d53598e23 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -208,5 +208,4 @@ public class CatalogUtils {
return 0;
}
}
-
}
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index a9bbdf6c45..2944ef3f6e 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -36,6 +36,10 @@ import java.util.function.Function;
import org.apache.ignite.configuration.NamedListView;
import org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
+import org.apache.ignite.internal.catalog.commands.DropIndexParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
import org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
@@ -59,6 +63,7 @@ import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.configuration.TablesView;
import org.apache.ignite.internal.schema.configuration.index.HashIndexChange;
+import org.apache.ignite.internal.schema.configuration.index.SortedIndexChange;
import org.apache.ignite.internal.schema.configuration.index.TableIndexChange;
import org.apache.ignite.internal.schema.configuration.index.TableIndexConfiguration;
import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
@@ -73,6 +78,7 @@ import org.apache.ignite.internal.util.IgniteSpinBusyLock;
import org.apache.ignite.internal.util.StringUtils;
import org.apache.ignite.lang.ErrorGroups;
import org.apache.ignite.lang.ErrorGroups.Common;
+import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteInternalException;
import org.apache.ignite.lang.IndexAlreadyExistsException;
import org.apache.ignite.lang.IndexNotFoundException;
@@ -96,6 +102,9 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
/** Table manager. */
private final TableManager tableManager;
+ /** Catalog manager. */
+ private final CatalogManager catalogManager;
+
/** Busy lock to stop synchronously. */
private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
@@ -106,17 +115,20 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
* Constructor.
*
* @param tablesCfg Tables and indexes configuration.
+ * @param catalogManager Catalog manager.
* @param schemaManager Schema manager.
* @param tableManager Table manager.
*/
public IndexManager(
TablesConfiguration tablesCfg,
+ CatalogManager catalogManager,
SchemaManager schemaManager,
TableManager tableManager
) {
this.tablesCfg = Objects.requireNonNull(tablesCfg, "tablesCfg");
this.schemaManager = Objects.requireNonNull(schemaManager, "schemaManager");
this.tableManager = tableManager;
+ this.catalogManager = catalogManager;
}
/** {@inheritDoc} */
@@ -142,8 +154,10 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
.toArray(String[]::new);
String pkName = table.name() + "_PK";
+ int pkId = table.tableId() + 1;
- return createIndexAsync("PUBLIC", pkName, table.name(), false,
+ // Update config bypassing the Catalog, because PK has just been created in Catalog with the table.
+ return createIndexInternal(pkId, "PUBLIC", pkName, table.name(), false,
change -> change.changeUniq(true).convert(HashIndexChange.class)
.changeColumnNames(pkColumns)
);
@@ -186,12 +200,104 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
* @param indexChange A consumer that suppose to change the configuration in order to provide description of an index.
* @return A future represented the result of creation.
*/
+ @Deprecated(forRemoval = true)
public CompletableFuture<Boolean> createIndexAsync(
String schemaName,
String indexName,
String tableName,
boolean failIfExists,
Consumer<TableIndexChange> indexChange
+ ) {
+ throw new UnsupportedOperationException("Method is no longer supported.");
+ }
+
+ /**
+ * Creates sorted index from provided parameters.
+ */
+ @Deprecated(forRemoval = true)
+ public CompletableFuture<Boolean> createSortedIndexAsync(CreateSortedIndexParams params) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new NodeStoppingException());
+ }
+
+ try {
+ Consumer<TableIndexChange> indexChanger = tableIndexChange -> {
+ tableIndexChange.convert(SortedIndexChange.class).changeColumns(colsInit -> {
+ for (int i = 0; i < params.columns().size(); i++) {
+ String columnName = params.columns().get(i);
+ CatalogColumnCollation collation = params.collations().get(i);
+ //TODO: https://issues.apache.org/jira/browse/IGNITE-17563 Pass null ordering for columns.
+ colsInit.create(columnName, colInit -> colInit.changeAsc(collation.asc()));
+ }
+ });
+ };
+
+ return catalogManager.createIndex(params)
+ .thenApply(ignore -> {
+ CatalogIndexDescriptor index = catalogManager.index(params.indexName(), Long.MAX_VALUE);
+ return index.id();
+ })
+ .thenCompose(indexId ->
+ createIndexInternal(
+ indexId,
+ params.schemaName(),
+ params.indexName(),
+ params.tableName(),
+ true,
+ indexChanger
+ )
+ );
+ } catch (Exception ex) {
+ return failedFuture(ex);
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ /**
+ * Creates hash index from provided parameters.
+ */
+ @Deprecated(forRemoval = true)
+ public CompletableFuture<Boolean> createHashIndexAsync(CreateHashIndexParams params) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new NodeStoppingException());
+ }
+
+ try {
+ Consumer<TableIndexChange> indexChanger = tableIndexChange -> {
+ tableIndexChange.convert(HashIndexChange.class)
+ .changeColumnNames(params.columns().toArray(STRING_EMPTY_ARRAY));
+ };
+
+ return catalogManager.createIndex(params)
+ .thenApply(ignore -> {
+ CatalogIndexDescriptor index = catalogManager.index(params.indexName(), Long.MAX_VALUE);
+ return index.id();
+ })
+ .thenCompose(indexId ->
+ createIndexInternal(
+ indexId,
+ params.schemaName(),
+ params.indexName(),
+ params.tableName(),
+ true,
+ indexChanger
+ )
+ );
+ } catch (Exception ex) {
+ return failedFuture(ex);
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ private CompletableFuture<Boolean> createIndexInternal(
+ int indexId,
+ String schemaName,
+ String indexName,
+ String tableName,
+ boolean failIfExists,
+ Consumer<TableIndexChange> indexChange
) {
if (!busyLock.enterBusy()) {
return failedFuture(new NodeStoppingException());
@@ -224,8 +330,6 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
int tableId = tableCfg.id();
- int indexId = tablesChange.globalIdCounter() + 1;
-
tablesChange.changeGlobalIdCounter(indexId);
Consumer<TableIndexChange> chg = indexChange.andThen(c -> c.changeTableId(tableId).changeId(indexId));
@@ -281,6 +385,33 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
String schemaName,
String indexName,
boolean failIfNotExists
+ ) {
+ throw new UnsupportedOperationException("Method is no longer supported.");
+ }
+
+ /**
+ * Drops the index with a given parameters asynchronously.
+ */
+ public CompletableFuture<Boolean> dropIndexAsync(DropIndexParams params) {
+ if (!busyLock.enterBusy()) {
+ throw new IgniteException(new NodeStoppingException());
+ }
+ try {
+ CompletableFuture<Boolean> future = catalogManager.dropIndex(params)
+ .thenCompose(ignore -> dropIndexAsyncInternal(params.schemaName(), params.indexName(), true));
+
+ future.whenComplete((res, ex) -> ex.printStackTrace());
+
+ return future;
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
+
+ private CompletableFuture<Boolean> dropIndexAsyncInternal(
+ String schemaName,
+ String indexName,
+ boolean failIfNotExists
) {
if (!busyLock.enterBusy()) {
return failedFuture(new NodeStoppingException());
@@ -325,6 +456,8 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
});
return future;
+ } catch (Exception ex) {
+ return failedFuture(ex);
} finally {
busyLock.leaveBusy();
}
diff --git a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
index 8230656614..40fb2c13f7 100644
--- a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
+++ b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
@@ -27,8 +27,10 @@ import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.notNullValue;
import static org.junit.jupiter.api.Assertions.fail;
+import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@@ -38,6 +40,12 @@ import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
+import org.apache.ignite.internal.catalog.commands.DropIndexParams;
+import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.configuration.testframework.ConfigurationExtension;
import org.apache.ignite.internal.configuration.testframework.InjectConfiguration;
import org.apache.ignite.internal.configuration.tree.ConverterToMapVisitor;
@@ -47,7 +55,6 @@ import org.apache.ignite.internal.index.event.IndexEventParameters;
import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
-import org.apache.ignite.internal.schema.configuration.index.SortedIndexChange;
import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
import org.apache.ignite.internal.table.InternalTable;
import org.apache.ignite.internal.table.TableImpl;
@@ -104,7 +111,16 @@ public class IndexManagerTest {
when(schManager.schemaRegistry(anyLong(), anyInt())).thenReturn(completedFuture(null));
- indexManager = new IndexManager(tablesConfig, schManager, tableManagerMock);
+ //TODO IGNITE-19082 drop mocked catalog manager.
+ CatalogManager catalogManager = mock(CatalogManager.class);
+ CatalogIndexDescriptor indexDescriptor = mock(CatalogIndexDescriptor.class);
+ when(catalogManager.createIndex(any(CreateHashIndexParams.class))).thenReturn(completedFuture(null));
+ when(catalogManager.createIndex(any(CreateSortedIndexParams.class))).thenReturn(completedFuture(null));
+ when(catalogManager.dropIndex(any())).thenReturn(completedFuture(null));
+ when(catalogManager.index(anyString(), anyLong())).thenReturn(indexDescriptor);
+ when(indexDescriptor.id()).thenReturn(1);
+
+ indexManager = new IndexManager(tablesConfig, catalogManager, schManager, tableManagerMock);
indexManager.start();
assertThat(
@@ -125,16 +141,14 @@ public class IndexManagerTest {
void configurationChangedWhenCreateIsInvoked() {
String indexName = "idx";
- assertThat(indexManager.createIndexAsync("sName", indexName, "tName", true, indexChange -> {
- SortedIndexChange sortedIndexChange = indexChange.convert(SortedIndexChange.class);
-
- sortedIndexChange.changeColumns(columns -> {
- columns.create("c1", columnChange -> columnChange.changeAsc(true));
- columns.create("c2", columnChange -> columnChange.changeAsc(false));
- });
-
- sortedIndexChange.changeTableId(tableId());
- }), willCompleteSuccessfully());
+ assertThat(indexManager.createSortedIndexAsync(
+ CreateSortedIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .tableName("tName")
+ .indexName(indexName)
+ .columns(List.of("c1", "c2"))
+ .collations(List.of(CatalogColumnCollation.ASC_NULLS_LAST, CatalogColumnCollation.DESC_NULLS_FIRST))
+ .build()), willCompleteSuccessfully());
var expected = List.of(
Map.of(
@@ -162,7 +176,24 @@ public class IndexManagerTest {
@Test
public void createIndexWithEmptyName() {
assertThat(
- indexManager.createIndexAsync("sName", "", "tName", true, indexChange -> {/* doesn't matter */}),
+ indexManager.createHashIndexAsync(
+ CreateHashIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .tableName("tName")
+ .indexName("")
+ .columns(List.of("c1"))
+ .build()),
+ willThrowFast(IgniteInternalException.class, "Index name should be at least 1 character long")
+ );
+
+ assertThat(
+ indexManager.createSortedIndexAsync(
+ CreateSortedIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .tableName("tName")
+ .indexName("")
+ .columns(List.of("c1"))
+ .build()),
willThrowFast(IgniteInternalException.class, "Index name should be at least 1 character long")
);
}
@@ -170,8 +201,12 @@ public class IndexManagerTest {
@Test
public void dropNonExistingIndex() {
assertThat(
- indexManager.dropIndexAsync("sName", "nonExisting", true),
- willThrowFast(IndexNotFoundException.class, "Index does not exist [name=\"sName\".\"nonExisting\"]")
+ indexManager.dropIndexAsync(
+ DropIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .indexName("nonExisting")
+ .build()),
+ willThrowFast(IndexNotFoundException.class, "Index does not exist [name=\"PUBLIC\".\"nonExisting\"]")
);
}
@@ -194,15 +229,15 @@ public class IndexManagerTest {
return completedFuture(true);
});
- assertThat(indexManager.createIndexAsync("sName", indexName, "tName", true, indexChange -> {
- SortedIndexChange sortedIndexChange = indexChange.convert(SortedIndexChange.class);
-
- sortedIndexChange.changeColumns(columns -> {
- columns.create("c2", columnChange -> columnChange.changeAsc(true));
- });
-
- sortedIndexChange.changeTableId(tableId());
- }), willCompleteSuccessfully());
+ assertThat(indexManager.createSortedIndexAsync(
+ CreateSortedIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .indexName(indexName)
+ .tableName("tName")
+ .columns(List.of("c2"))
+ .collations(List.of(CatalogColumnCollation.ASC_NULLS_LAST))
+ .build()),
+ willCompleteSuccessfully());
List<Integer> indexIds = tablesConfig.indexes().value().stream()
.map(TableIndexView::id)
@@ -217,7 +252,12 @@ public class IndexManagerTest {
assertThat(holder.get().tableId(), equalTo(tableId()));
assertThat(holder.get().indexDescriptor().name(), equalTo(indexName));
- assertThat(indexManager.dropIndexAsync("sName", indexName, true), willCompleteSuccessfully());
+ assertThat(indexManager.dropIndexAsync(
+ DropIndexParams.builder()
+ .schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
+ .indexName(indexName)
+ .build()),
+ willCompleteSuccessfully());
assertThat(holder.get(), notNullValue());
assertThat(holder.get().indexId(), equalTo(indexId));
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
index e511cac7c0..3c7eca2188 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
@@ -23,12 +23,14 @@ import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasItems;
import static org.hamcrest.Matchers.notNullValue;
+import java.util.List;
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.Ignite;
import org.apache.ignite.internal.app.IgniteImpl;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.DropIndexParams;
import org.apache.ignite.internal.index.event.IndexEvent;
import org.apache.ignite.internal.index.event.IndexEventParameters;
-import org.apache.ignite.internal.schema.configuration.index.HashIndexChange;
import org.apache.ignite.internal.sql.engine.ClusterPerClassIntegrationTest;
import org.apache.ignite.internal.table.TableImpl;
import org.apache.ignite.internal.testframework.WorkDirectoryExtension;
@@ -68,12 +70,13 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
CompletableFuture<IndexEventParameters> indexCreatedFuture = registerListener(indexManager, IndexEvent.CREATE);
- await(indexManager.createIndexAsync(
- "PUBLIC",
- "INAME",
- "TNAME",
- true,
- tableIndexChange -> tableIndexChange.convert(HashIndexChange.class).changeColumnNames("C3", "C2")
+ await(indexManager.createHashIndexAsync(
+ CreateHashIndexParams.builder()
+ .schemaName("PUBLIC")
+ .indexName("INAME")
+ .tableName("TNAME")
+ .columns(List.of("C3", "C2"))
+ .build()
));
int createdIndexId;
@@ -90,7 +93,11 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
CompletableFuture<IndexEventParameters> indexDroppedFuture = registerListener(indexManager, IndexEvent.DROP);
- await(indexManager.dropIndexAsync("PUBLIC", "INAME", true));
+ await(indexManager.dropIndexAsync(
+ DropIndexParams.builder()
+ .schemaName("PUBLIC")
+ .indexName("INAME")
+ .build()));
{
IndexEventParameters params = await(indexDroppedFuture);
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 30dbdd0552..60e2e37a26 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -386,7 +386,7 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
null
);
- var indexManager = new IndexManager(tablesConfig, schemaManager, tableManager);
+ var indexManager = new IndexManager(tablesConfig, catalogManager, schemaManager, tableManager);
SqlQueryProcessor qryEngine = new SqlQueryProcessor(
registry,
@@ -1205,9 +1205,9 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
*/
private void createTableWithData(List<IgniteImpl> nodes, String name, int replicas, int partitions) {
try (Session session = nodes.get(0).sql().createSession()) {
- session.execute(null,
- String.format("CREATE ZONE IF NOT EXISTS ZONE_%s WITH REPLICAS=%d, PARTITIONS=%d", name, replicas, partitions));
- session.execute(null, "CREATE TABLE IF NOT EXISTS " + name
+ session.execute(null, String.format("CREATE ZONE IF NOT EXISTS ZONE_%s WITH REPLICAS=%d, PARTITIONS=%d",
+ name.toUpperCase(), replicas, partitions));
+ session.execute(null, "CREATE TABLE IF NOT EXISTS " + name.toUpperCase()
+ "(id INT PRIMARY KEY, name VARCHAR) WITH PRIMARY_ZONE='ZONE_" + name.toUpperCase() + "';");
for (int i = 0; i < 100; i++) {
diff --git a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
index d38a1597e8..a9cd3b548b 100644
--- a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
+++ b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
@@ -550,7 +550,7 @@ public class IgniteImpl implements Ignite {
distributionZoneManager
);
- indexManager = new IndexManager(tablesConfig, schemaManager, distributedTblMgr);
+ indexManager = new IndexManager(tablesConfig, catalogManager, schemaManager, distributedTblMgr);
qryEngine = new SqlQueryProcessor(
registry,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index f56848c215..5760dc2a85 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -106,7 +106,7 @@ public class DdlCommandHandler {
protected final TableManager tableManager;
- private final IndexManager indexManager;
+ protected final IndexManager indexManager;
private final DataStorageManager dataStorageManager;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
index bb702d4566..7fbd884f9b 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
@@ -17,6 +17,9 @@
package org.apache.ignite.internal.sql.engine.exec.ddl;
+import static java.util.concurrent.CompletableFuture.failedFuture;
+import static org.apache.ignite.lang.ErrorGroups.Sql.UNSUPPORTED_DDL_OPERATION_ERR;
+
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.catalog.CatalogManager;
@@ -35,6 +38,7 @@ import org.apache.ignite.internal.sql.engine.prepare.ddl.DropIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropTableCommand;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.table.distributed.TableManager;
+import org.apache.ignite.lang.IgniteInternalCheckedException;
import org.apache.ignite.lang.IndexAlreadyExistsException;
import org.apache.ignite.lang.IndexNotFoundException;
import org.apache.ignite.lang.TableAlreadyExistsException;
@@ -86,28 +90,25 @@ public class DdlCommandHandlerWrapper extends DdlCommandHandler {
} else if (cmd instanceof AlterColumnCommand) {
return catalogManager.alterColumn(DdlToCatalogCommandConverter.convert((AlterColumnCommand) cmd))
.handle(handleModificationResult(((AlterColumnCommand) cmd).ifTableExists(), TableNotFoundException.class));
- }
+ } else if (cmd instanceof CreateIndexCommand) {
+ AbstractIndexCommandParams params = DdlToCatalogCommandConverter.convert((CreateIndexCommand) cmd);
- // Handle command in usual way.
- CompletableFuture<Boolean> ddlCommandFuture = super.handle(cmd);
+ if (params instanceof CreateSortedIndexParams) {
+ return indexManager.createSortedIndexAsync((CreateSortedIndexParams) params)
+ .handle(handleModificationResult(((CreateIndexCommand) cmd).ifNotExists(), IndexAlreadyExistsException.class));
+ } else if (params instanceof CreateHashIndexParams) {
+ return indexManager.createHashIndexAsync((CreateHashIndexParams) params)
+ .handle(handleModificationResult(((CreateIndexCommand) cmd).ifNotExists(), IndexAlreadyExistsException.class));
+ }
- if (cmd instanceof CreateIndexCommand) {
- return ddlCommandFuture
- .thenCompose(res -> {
- AbstractIndexCommandParams params = DdlToCatalogCommandConverter.convert((CreateIndexCommand) cmd);
- if (params instanceof CreateSortedIndexParams) {
- return catalogManager.createIndex((CreateSortedIndexParams) params);
- } else {
- return catalogManager.createIndex((CreateHashIndexParams) params);
- }
- }).handle(handleModificationResult(((CreateIndexCommand) cmd).ifNotExists(), IndexAlreadyExistsException.class));
+ return failedFuture(new IgniteInternalCheckedException(UNSUPPORTED_DDL_OPERATION_ERR, "Unsupported DDL operation ["
+ + "cmdName=" + cmd.getClass().getSimpleName() + "]"));
} else if (cmd instanceof DropIndexCommand) {
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.dropIndex(DdlToCatalogCommandConverter.convert((DropIndexCommand) cmd))
- .handle(handleModificationResult(((DropIndexCommand) cmd).ifNotExists(), IndexNotFoundException.class))
- );
+ return indexManager.dropIndexAsync(DdlToCatalogCommandConverter.convert((DropIndexCommand) cmd))
+ .handle(handleModificationResult(((DropIndexCommand) cmd).ifNotExists(), IndexNotFoundException.class));
}
- return ddlCommandFuture;
+ // Handle other commands in usual way.
+ return super.handle(cmd);
}
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index 9aa9471037..955a4223c2 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -48,6 +48,9 @@ import java.util.function.Consumer;
import java.util.function.LongFunction;
import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologySnapshot;
@@ -284,11 +287,17 @@ public class MockedStructuresTest extends IgniteAbstractTest {
//TODO IGNITE-19082 drop mocked catalog manager.
catalogManager = mock(CatalogManager.class);
- CatalogTableDescriptor descriptor = mock(CatalogTableDescriptor.class);
- when(descriptor.id()).thenReturn(1);
+ CatalogTableDescriptor tableDescriptor = mock(CatalogTableDescriptor.class);
+ CatalogIndexDescriptor indexDescriptor = mock(CatalogIndexDescriptor.class);
+ when(tableDescriptor.id()).thenReturn(1);
+ when(indexDescriptor.id()).thenReturn(1);
when(catalogManager.createTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.createIndex(any(CreateHashIndexParams.class))).thenReturn(completedFuture(null));
+ when(catalogManager.createIndex(any(CreateSortedIndexParams.class))).thenReturn(completedFuture(null));
when(catalogManager.dropTable(any())).thenReturn(completedFuture(null));
- when(catalogManager.table(anyString(), anyLong())).thenReturn(descriptor);
+ when(catalogManager.dropIndex(any())).thenReturn(completedFuture(null));
+ when(catalogManager.table(anyString(), anyLong())).thenReturn(tableDescriptor);
+ when(catalogManager.index(anyString(), anyLong())).thenReturn(indexDescriptor);
cmgMgr = mock(ClusterManagementGroupManager.class);
@@ -304,7 +313,7 @@ public class MockedStructuresTest extends IgniteAbstractTest {
tblManager = mockManagers();
- idxManager = new IndexManager(tblsCfg, schemaManager, tblManager);
+ idxManager = new IndexManager(tblsCfg, catalogManager, schemaManager, tblManager);
idxManager.start();
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 375a1fa542..6a7b8b7721 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -1746,7 +1746,13 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
}
- /** See {@link #alterTableAsync(String, Function)} for details. */
+ /**
+ * Alter table configuration.
+ *
+ * @see AlterTableAddColumnParams
+ * @see AlterTableDropColumnParams
+ */
+ @Deprecated(forRemoval = true)
private CompletableFuture<Void> alterTableAsyncInternal(String name, Function<TableChange, Boolean> tableChange) {
CompletableFuture<Void> tblFut = new CompletableFuture<>();
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index 883c0bf771..83b5e6f5ff 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -68,7 +68,10 @@ import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
+import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
+import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.cluster.management.ClusterManagementGroupManager;
@@ -861,10 +864,16 @@ public class TableManagerTest extends IgniteAbstractTest {
//TODO IGNITE-19082 drop mocked catalog manager.
CatalogManager catalogManager = mock(CatalogManager.class);
CatalogTableDescriptor tableDescriptor = mock(CatalogTableDescriptor.class);
+ CatalogIndexDescriptor indexDescriptor = mock(CatalogIndexDescriptor.class);
+ when(tableDescriptor.id()).thenReturn(1);
+ when(indexDescriptor.id()).thenReturn(1);
when(catalogManager.createTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.createIndex(any(CreateHashIndexParams.class))).thenReturn(completedFuture(null));
+ when(catalogManager.createIndex(any(CreateSortedIndexParams.class))).thenReturn(completedFuture(null));
when(catalogManager.dropTable(any())).thenReturn(completedFuture(null));
+ when(catalogManager.dropIndex(any())).thenReturn(completedFuture(null));
when(catalogManager.table(anyString(), anyLong())).thenReturn(tableDescriptor);
- when(tableDescriptor.id()).thenReturn(1);
+ when(catalogManager.index(anyString(), anyLong())).thenReturn(indexDescriptor);
TableManager tableManager = new TableManager(
"test",
[ignite-3] 16/19: Calculate index storage descriptors synchronously on table start.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 9a55705ccf1f34793a6f3aac91a3b4e8cd963868
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 21 16:29:04 2023 +0300
Calculate index storage descriptors synchronously on table start.
---
.../apache/ignite/internal/schema/SchemaUtils.java | 2 -
.../sql/engine/exec/MockedStructuresTest.java | 1 -
.../index/StorageIndexDescriptorSupplier.java | 22 ++++
.../internal/table/distributed/TableManager.java | 117 +++++----------------
.../table/distributed/TableManagerTest.java | 6 +-
5 files changed, 54 insertions(+), 94 deletions(-)
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
index ac4cef9e78..a81498ad94 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaUtils.java
@@ -19,8 +19,6 @@ package org.apache.ignite.internal.schema;
import java.util.Arrays;
import java.util.Comparator;
-import org.apache.ignite.internal.schema.configuration.ConfigurationToSchemaDescriptorConverter;
-import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.mapping.ColumnMapper;
import org.apache.ignite.internal.schema.mapping.ColumnMapping;
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index aea8087ad1..209a911f6a 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -33,7 +33,6 @@ import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.when;
import java.util.Collection;
diff --git a/modules/storage-api/src/main/java/org/apache/ignite/internal/storage/index/StorageIndexDescriptorSupplier.java b/modules/storage-api/src/main/java/org/apache/ignite/internal/storage/index/StorageIndexDescriptorSupplier.java
index 4034d69210..590508ca09 100644
--- a/modules/storage-api/src/main/java/org/apache/ignite/internal/storage/index/StorageIndexDescriptorSupplier.java
+++ b/modules/storage-api/src/main/java/org/apache/ignite/internal/storage/index/StorageIndexDescriptorSupplier.java
@@ -22,6 +22,12 @@ import static org.apache.ignite.internal.schema.CatalogDescriptorUtils.toTableDe
import static org.apache.ignite.internal.schema.configuration.SchemaConfigurationUtils.findIndexView;
import static org.apache.ignite.internal.schema.configuration.SchemaConfigurationUtils.findTableView;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.configuration.TablesView;
@@ -34,6 +40,7 @@ import org.jetbrains.annotations.Nullable;
// TODO: IGNITE-19717 Get rid of it
public class StorageIndexDescriptorSupplier {
private final TablesConfiguration tablesConfig;
+ private final Map<Integer, StorageIndexDescriptor> indexDescriptors;
/**
* Constructor.
@@ -42,6 +49,17 @@ public class StorageIndexDescriptorSupplier {
*/
public StorageIndexDescriptorSupplier(TablesConfiguration tablesConfig) {
this.tablesConfig = tablesConfig;
+ this.indexDescriptors = null;
+ }
+
+ /**
+ * Constructor.
+ */
+ public StorageIndexDescriptorSupplier(CatalogTableDescriptor tableDescriptor, List<CatalogIndexDescriptor> indexDescriptors) {
+ this.tablesConfig = null;
+ this.indexDescriptors = indexDescriptors.stream()
+ .map(idx -> StorageIndexDescriptor.create(tableDescriptor, idx))
+ .collect(Collectors.toMap(StorageIndexDescriptor::id, Function.identity()));
}
/**
@@ -51,6 +69,10 @@ public class StorageIndexDescriptorSupplier {
*/
@Nullable
public StorageIndexDescriptor get(int id) {
+ if (tablesConfig == null) {
+ return indexDescriptors.get(id);
+ }
+
TablesView tablesView = tablesConfig.value();
TableIndexView indexView = findIndexView(tablesView, id);
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 3d269271fe..ee926b253d 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -42,7 +42,6 @@ import static org.apache.ignite.internal.utils.RebalanceUtil.extractPartitionNum
import static org.apache.ignite.internal.utils.RebalanceUtil.extractTableId;
import static org.apache.ignite.internal.utils.RebalanceUtil.pendingPartAssignmentsKey;
import static org.apache.ignite.internal.utils.RebalanceUtil.stablePartAssignmentsKey;
-import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
import static org.apache.ignite.lang.IgniteStringFormatter.format;
import it.unimi.dsi.fastutil.ints.Int2ObjectOpenHashMap;
@@ -90,10 +89,10 @@ import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
import org.apache.ignite.internal.catalog.commands.AlterTableDropColumnParams;
-import org.apache.ignite.internal.catalog.commands.ColumnParams;
import org.apache.ignite.internal.catalog.commands.CreateTableParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogObjectDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
@@ -137,7 +136,6 @@ import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.SchemaRegistry;
import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
-import org.apache.ignite.internal.schema.configuration.PrimaryKeyView;
import org.apache.ignite.internal.schema.configuration.TableChange;
import org.apache.ignite.internal.schema.configuration.TableConfiguration;
import org.apache.ignite.internal.schema.configuration.TablesChange;
@@ -191,8 +189,6 @@ import org.apache.ignite.internal.util.PendingComparableValuesTracker;
import org.apache.ignite.internal.utils.RebalanceUtil;
import org.apache.ignite.internal.vault.VaultManager;
import org.apache.ignite.lang.ByteArray;
-import org.apache.ignite.lang.ColumnAlreadyExistsException;
-import org.apache.ignite.lang.ColumnNotFoundException;
import org.apache.ignite.lang.DistributionZoneNotFoundException;
import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteExceptionUtils;
@@ -206,7 +202,6 @@ import org.apache.ignite.network.ClusterService;
import org.apache.ignite.network.MessagingService;
import org.apache.ignite.network.TopologyService;
import org.apache.ignite.raft.jraft.storage.impl.VolatileRaftMetaStorage;
-import org.apache.ignite.sql.SqlException;
import org.apache.ignite.table.Table;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
@@ -602,16 +597,15 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
assert !assignments.isEmpty() : "Couldn't create the table with empty assignments.";
- int[] indexesToWait = Arrays.stream(catalogManager.schema(evt.catalogVersion()).indexes())
+ List<CatalogIndexDescriptor> indexDescriptors = Arrays.stream(catalogManager.schema(evt.catalogVersion()).indexes())
.filter(i -> i.tableId() == tableId)
- .mapToInt(CatalogObjectDescriptor::id)
- .toArray();
+ .collect(toList());
CompletableFuture<?> createTableFut = createTableLocally(
evt.causalityToken(),
tableDescriptor,
zoneDescriptor,
- indexesToWait,
+ indexDescriptors,
assignments
).whenComplete((v, e) -> {
if (e == null) {
@@ -731,6 +725,8 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
String localMemberName = localNode().name();
+ CompletableFuture<SchemaRegistry> schemaRegistryFuture = schemaManager.schemaRegistry(causalityToken, tableId);
+
// Create new raft nodes according to new assignments.
Supplier<CompletableFuture<Void>> updateAssignmentsClosure = () -> {
for (int i = 0; i < partitions; i++) {
@@ -888,7 +884,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
txStateStorage,
partitionUpdateHandlers,
updatedRaftGroupService,
- schemaManager.schemaRegistry(causalityToken, tableId)
+ schemaRegistryFuture
);
} catch (NodeStoppingException ex) {
throw new AssertionError("Loza was stopped before Table manager", ex);
@@ -1225,14 +1221,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param causalityToken Causality token.
* @param tableDescriptor Catalog table descriptor.
* @param zoneDescriptor Catalog distributed zone descriptor.
- * @param tableIndexes Ids of indexes belongs to the table.
+ * @param indexDescriptors Catalog index descriptors for given table.
* @return Future that will be completed when local changes related to the table creation are applied.
*/
private CompletableFuture<?> createTableLocally(
long causalityToken,
CatalogTableDescriptor tableDescriptor,
CatalogZoneDescriptor zoneDescriptor,
- int[] tableIndexes,
+ List<CatalogIndexDescriptor> indexDescriptors,
List<Set<Assignment>> assignments
) {
String tableName = tableDescriptor.name();
@@ -1240,7 +1236,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
LOG.trace("Creating local table: name={}, id={}, token={}", tableDescriptor.name(), tableDescriptor.id(), causalityToken);
- MvTableStorage tableStorage = createTableStorage(tableDescriptor, zoneDescriptor);
+ MvTableStorage tableStorage = createTableStorage(tableDescriptor, zoneDescriptor, indexDescriptors);
TxStateTableStorage txStateStorage = createTxStateTableStorage(tableDescriptor, zoneDescriptor);
int partitions = zoneDescriptor.partitions();
@@ -1253,7 +1249,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
var table = new TableImpl(internalTable, lockMgr);
// TODO: IGNITE-19082 Need another way to wait for indexes
- table.addIndexesToWait(tableIndexes);
+ table.addIndexesToWait(indexDescriptors.stream().mapToInt(CatalogObjectDescriptor::id).toArray());
tablesByIdVv.update(causalityToken, (previous, e) -> inBusyLock(busyLock, () -> {
if (e != null) {
@@ -1290,6 +1286,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
// TODO: investigate why createParts future hangs.
+ // return createPartsFut;
return completedFuture(false);
}
@@ -1298,8 +1295,10 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
*
* @param tableDescriptor Catalog table descriptor.
* @param zoneDescriptor Catalog distributed zone descriptor.
+ * @param indexDescriptors Index descriptors.
*/
- protected MvTableStorage createTableStorage(CatalogTableDescriptor tableDescriptor, CatalogZoneDescriptor zoneDescriptor) {
+ protected MvTableStorage createTableStorage(CatalogTableDescriptor tableDescriptor, CatalogZoneDescriptor zoneDescriptor,
+ List<CatalogIndexDescriptor> indexDescriptors) {
CatalogDataStorageDescriptor dataStorage = zoneDescriptor.getDataStorage();
StorageEngine engine = dataStorageMgr.engine(dataStorage.getEngine());
@@ -1308,7 +1307,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
MvTableStorage tableStorage = engine.createMvTable(
new StorageTableDescriptor(tableDescriptor.id(), zoneDescriptor.partitions(), dataStorage.getDataRegion()),
- new StorageIndexDescriptorSupplier(tablesCfg)
+ new StorageIndexDescriptorSupplier(tableDescriptor, indexDescriptors)
);
tableStorage.start();
@@ -1678,8 +1677,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
throw new IgniteException(new NodeStoppingException());
}
try {
- return catalogManager.addColumn(params)
- .thenCompose(ignore -> addColumnInternal(params.tableName(), params.columns()));
+ return catalogManager.addColumn(params);
} finally {
busyLock.leaveBusy();
}
@@ -1697,8 +1695,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
throw new IgniteException(new NodeStoppingException());
}
try {
- return catalogManager.dropColumn(params)
- .thenCompose(ignore -> dropColumnInternal(params.tableName(), params.columns()));
+ return catalogManager.dropColumn(params);
} finally {
busyLock.leaveBusy();
}
@@ -2770,78 +2767,20 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
return zone.replicas();
}
- // Copied from DdlCommandHandler
@Deprecated(forRemoval = true)
- private CompletableFuture<Void> addColumnInternal(String fullName, List<ColumnParams> columnParams) {
- return alterTableAsyncInternal(
- fullName,
- chng -> {
- if (columnParams.isEmpty()) {
- return false;
- }
-
- chng.changeColumns(cols -> {
- Set<String> colNamesToOrders = new HashSet<>(cols.namedListKeys());
-
- columnParams.stream()
- .filter(k -> colNamesToOrders.contains(k.name()))
- .findAny()
- .ifPresent(c -> {
- throw new ColumnAlreadyExistsException(c.name());
- });
-
- for (ColumnParams col : columnParams) {
- cols.create(col.name(), colChg -> CatalogDescriptorUtils.convertColumnDefinition(col, colChg));
- }
- });
-
- return true;
- }
- );
- }
-
- // Copied from DdlCommandHandler
- // TODO: IGNITE-19082 Drop unused temporary method.
- @Deprecated(forRemoval = true)
- private CompletableFuture<Void> dropColumnInternal(String tableName, Set<String> colNames) {
- AtomicBoolean ret = new AtomicBoolean(true);
-
- return alterTableAsyncInternal(
- tableName,
- chng -> {
- chng.changeColumns(cols -> {
- ret.set(true); // Reset state if closure have been restarted.
+ private static CatalogDataStorageDescriptor toDataStorageDescriptor(DataStorageView config) {
+ String dataRegion;
- PrimaryKeyView priKey = chng.primaryKey();
-
- Set<String> colNamesToOrders = new HashSet<>(cols.namedListKeys());
-
- Set<String> colNames0 = new HashSet<>();
-
- Set<String> primaryCols = Set.of(priKey.columns());
-
- // Catalog verification passe, so we can omit validation here.
- // reportIndexedColumns(tableName, colNames, primaryCols);
-
- for (String colName : colNames) {
- if (!colNamesToOrders.contains(colName)) {
- ret.set(false);
-
- throw new ColumnNotFoundException(DEFAULT_SCHEMA_NAME, tableName, colName);
- } else {
- colNames0.add(colName);
- }
+ try {
+ Method dataRegionMethod = config.getClass().getMethod("dataRegion");
- if (primaryCols.contains(colName)) {
- throw new SqlException(STMT_VALIDATION_ERR,
- format("Can`t delete column, belongs to primary key: [name={}]", colName));
- }
- }
+ dataRegionMethod.setAccessible(true);
- colNames0.forEach(cols::delete);
- });
+ dataRegion = (String) dataRegionMethod.invoke(config);
+ } catch (ReflectiveOperationException e) {
+ dataRegion = e.getMessage();
+ }
- return ret.get();
- });
+ return new CatalogDataStorageDescriptor(config.name(), dataRegion);
}
}
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index c5057c465b..c4458f4a6a 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -69,6 +69,7 @@ import org.apache.ignite.internal.catalog.CatalogServiceImpl;
import org.apache.ignite.internal.catalog.ClockWaiter;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.catalog.storage.UpdateLogImpl;
@@ -887,8 +888,9 @@ public class TableManagerTest extends IgniteAbstractTest {
) {
@Override
- protected MvTableStorage createTableStorage(CatalogTableDescriptor tableDescriptor, CatalogZoneDescriptor zoneDescriptor) {
- mvTableStorage = spy(super.createTableStorage(tableDescriptor, zoneDescriptor));
+ protected MvTableStorage createTableStorage(CatalogTableDescriptor tableDescriptor, CatalogZoneDescriptor zoneDescriptor,
+ List<CatalogIndexDescriptor> indexDescriptors) {
+ mvTableStorage = spy(super.createTableStorage(tableDescriptor, zoneDescriptor, indexDescriptors));
tableStorageDecorator.accept(mvTableStorage);
[ignite-3] 19/19: Resolve conflicts
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit a4f076292182a900b43ac17f55335ccf9b01e2d8
Author: amashenkov <an...@gmail.com>
AuthorDate: Thu Jul 27 19:50:04 2023 +0300
Resolve conflicts
---
.../rebalance/RebalanceUtilUpdateAssignmentsTest.java | 1 +
.../org/apache/ignite/internal/schema/SchemaManager.java | 5 -----
.../ignite/internal/sql/engine/SqlQueryProcessor.java | 5 -----
.../internal/sql/engine/exec/ddl/DdlCommandHandler.java | 13 +------------
.../internal/sql/engine/util/HashFunctionFactoryImpl.java | 7 ++++---
.../exec/ddl/DdlCommandHandlerExceptionHandlingTest.java | 8 +-------
.../exec/ddl/DistributionZoneDdlCommandHandlerTest.java | 8 +-------
.../ignite/internal/table/distributed/TableManager.java | 2 --
8 files changed, 8 insertions(+), 41 deletions(-)
diff --git a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/rebalance/RebalanceUtilUpdateAssignmentsTest.java b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/rebalance/RebalanceUtilUpdateAssignmentsTest.java
index 93800f158e..2bdfe4c0e7 100644
--- a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/rebalance/RebalanceUtilUpdateAssignmentsTest.java
+++ b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/rebalance/RebalanceUtilUpdateAssignmentsTest.java
@@ -97,6 +97,7 @@ public class RebalanceUtilUpdateAssignmentsTest extends IgniteAbstractTest {
private final CatalogTableDescriptor tableDescriptor = new CatalogTableDescriptor(
1,
"table1",
+ 1,
0,
List.of(new CatalogTableColumnDescriptor("k1", ColumnType.INT32, false, 0, 0, 0, null)),
List.of("k1"),
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index f383a8f940..f6bf76b699 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -44,15 +44,10 @@ import org.apache.ignite.internal.catalog.events.CatalogEventParameters;
import org.apache.ignite.internal.catalog.events.CreateTableEventParameters;
import org.apache.ignite.internal.catalog.events.DropColumnEventParameters;
import org.apache.ignite.internal.causality.IncrementalVersionedValue;
-import org.apache.ignite.internal.logger.IgniteLogger;
-import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.manager.IgniteComponent;
import org.apache.ignite.internal.manager.Producer;
import org.apache.ignite.internal.metastorage.Entry;
import org.apache.ignite.internal.metastorage.MetaStorageManager;
-import org.apache.ignite.internal.schema.configuration.ColumnView;
-import org.apache.ignite.internal.schema.configuration.ExtendedTableView;
-import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.event.SchemaEvent;
import org.apache.ignite.internal.schema.event.SchemaEventParameters;
import org.apache.ignite.internal.schema.marshaller.schema.SchemaSerializerImpl;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index c0bb72e478..3066fdabb6 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -141,8 +141,6 @@ public class SqlQueryProcessor implements QueryProcessor {
private final SchemaManager schemaManager;
- private final Consumer<LongFunction<CompletableFuture<?>>> registry;
-
private final DataStorageManager dataStorageManager;
private final Supplier<Map<String, Map<String, Class<?>>>> dataStorageFieldsSupplier;
@@ -192,7 +190,6 @@ public class SqlQueryProcessor implements QueryProcessor {
HybridClock clock,
CatalogManager catalogManager
) {
- this.registry = registry;
this.clusterSrvc = clusterSrvc;
this.tableManager = tableManager;
this.indexManager = indexManager;
@@ -240,8 +237,6 @@ public class SqlQueryProcessor implements QueryProcessor {
var ddlCommandHandler = new DdlCommandHandler(
distributionZoneManager,
- tableManager,
- indexManager,
dataStorageManager,
catalogManager
);
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index ce5daac344..ddda335276 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -31,7 +31,6 @@ import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
-import org.apache.ignite.internal.index.IndexManager;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AbstractTableDdlCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterColumnCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableAddCommand;
@@ -46,7 +45,6 @@ import org.apache.ignite.internal.sql.engine.prepare.ddl.DropIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropTableCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropZoneCommand;
import org.apache.ignite.internal.storage.DataStorageManager;
-import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.internal.util.StringUtils;
import org.apache.ignite.lang.DistributionZoneAlreadyExistsException;
import org.apache.ignite.lang.DistributionZoneNotFoundException;
@@ -60,10 +58,6 @@ import org.apache.ignite.sql.SqlException;
public class DdlCommandHandler {
private final DistributionZoneManager distributionZoneManager;
- private final TableManager tableManager;
-
- private final IndexManager indexManager;
-
private final DataStorageManager dataStorageManager;
private final CatalogManager catalogManager;
@@ -73,14 +67,10 @@ public class DdlCommandHandler {
*/
public DdlCommandHandler(
DistributionZoneManager distributionZoneManager,
- TableManager tableManager,
- IndexManager indexManager,
DataStorageManager dataStorageManager,
CatalogManager catalogManager
) {
this.distributionZoneManager = distributionZoneManager;
- this.tableManager = tableManager;
- this.indexManager = indexManager;
this.dataStorageManager = dataStorageManager;
this.catalogManager = catalogManager;
}
@@ -277,8 +267,7 @@ public class DdlCommandHandler {
if (params instanceof CreateSortedIndexParams) {
return catalogManager.createIndex((CreateSortedIndexParams) params)
.handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
- }
- else {
+ } else {
return catalogManager.createIndex((CreateHashIndexParams) params)
.handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
index 6f583b43b6..3ae492d834 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/HashFunctionFactoryImpl.java
@@ -37,19 +37,20 @@ import org.apache.ignite.internal.util.HashCalculator;
* Factory for creating a function to calculate the hash of the specified fields of a row.
*/
public class HashFunctionFactoryImpl<T> implements HashFunctionFactory<T> {
- private final SqlSchemaManager sqlSchemaManager;
private final IgniteCatalogSchema sqlSchema;
private final RowHandler<T> rowHandler;
+ /** Constructor. */
@Deprecated(forRemoval = true)
public HashFunctionFactoryImpl(SqlSchemaManager sqlSchemaManager, RowHandler<T> rowHandler) {
- this.sqlSchemaManager = sqlSchemaManager;
+ assert false;
+
this.rowHandler = rowHandler;
this.sqlSchema = null;
}
+ /** Constructor. */
public HashFunctionFactoryImpl(IgniteCatalogSchema sqlSchema, RowHandler<T> rowHandler) {
- this.sqlSchemaManager = null;
this.rowHandler = rowHandler;
this.sqlSchema = sqlSchema;
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
index 26bf4c8f28..8b19825e86 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
@@ -69,12 +69,6 @@ import org.mockito.quality.Strictness;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
public class DdlCommandHandlerExceptionHandlingTest extends IgniteAbstractTest {
- @Mock
- private TableManager tableManager;
-
- @Mock
- private IndexManager indexManager;
-
@Mock
private DataStorageManager dataStorageManager;
@@ -125,7 +119,7 @@ public class DdlCommandHandlerExceptionHandlingTest extends IgniteAbstractTest {
"node"
);
- commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager, catalogManager);
+ commandHandler = new DdlCommandHandler(distributionZoneManager, dataStorageManager, catalogManager);
}
@AfterEach
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
index d0a057b241..267db57322 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
@@ -57,12 +57,6 @@ public class DistributionZoneDdlCommandHandlerTest extends IgniteAbstractTest {
/** Holder of the result of the invoked method. */
private final AtomicReference<Object[]> invocationResultHolder = new AtomicReference<>();
- @Mock
- private TableManager tableManager;
-
- @Mock
- private IndexManager indexManager;
-
@Mock
private DataStorageManager dataStorageManager;
@@ -83,7 +77,7 @@ public class DistributionZoneDdlCommandHandlerTest extends IgniteAbstractTest {
return CompletableFuture.completedFuture(null);
});
- commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager, catalogManager);
+ commandHandler = new DdlCommandHandler(distributionZoneManager, dataStorageManager, catalogManager);
}
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index b923703efe..ae62851f3c 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -28,7 +28,6 @@ import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.causality.IncrementalVersionedValue.dependingOn;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.partitionAssignments;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.tableAssignments;
-import static org.apache.ignite.internal.distributionzones.rebalance.ZoneCatalogDescriptorUtils.toZoneDescriptor;
import static org.apache.ignite.internal.metastorage.dsl.Operations.put;
import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
import static org.apache.ignite.internal.util.IgniteUtils.shutdownAndAwaitTermination;
@@ -82,7 +81,6 @@ import org.apache.ignite.internal.affinity.Assignment;
import org.apache.ignite.internal.baseline.BaselineManager;
import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
-import org.apache.ignite.internal.catalog.commands.AlterTableDropColumnParams;
import org.apache.ignite.internal.catalog.commands.CreateTableParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
[ignite-3] 12/19: wip. Use catalog version for cache plans.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 06d31d44af36f0b122c2723ee079d9a8112e37a2
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 21 02:19:07 2023 +0300
wip. Use catalog version for cache plans.
---
.../apache/ignite/internal/index/IndexManager.java | 1 -
.../internal/sql/engine/SqlQueryProcessor.java | 3 +--
.../sql/engine/exec/ExecutableTableRegistry.java | 3 ++-
.../engine/exec/ExecutableTableRegistryImpl.java | 30 ++++++++++------------
.../exec/ExecutionDependencyResolverImpl.java | 2 +-
.../sql/engine/exec/LogicalRelImplementor.java | 24 ++++++++---------
.../internal/sql/engine/prepare/CacheKey.java | 28 ++++++++++----------
.../sql/engine/prepare/PrepareServiceImpl.java | 4 ++-
.../exec/ExecutableTableRegistrySelfTest.java | 2 +-
.../exec/ExecutionDependencyResolverSelfTest.java | 16 ++++++------
.../engine/exec/NoOpExecutableTableRegistry.java | 2 +-
.../internal/table/distributed/TableManager.java | 9 +++----
12 files changed, 59 insertions(+), 65 deletions(-)
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index 7e5caf9491..4a4853c2b8 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -17,7 +17,6 @@
package org.apache.ignite.internal.index;
-import static java.util.concurrent.CompletableFuture.allOf;
import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import static org.apache.ignite.internal.util.ArrayUtils.STRING_EMPTY_ARRAY;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index 4f64f709d3..c0bb72e478 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -250,8 +250,7 @@ public class SqlQueryProcessor implements QueryProcessor {
var dependencyResolver = new ExecutionDependencyResolverImpl(executableTableRegistry);
- CatalogSqlSchemaManager sqlSchemaManager = new CatalogSqlSchemaManager(catalogManager, 1000);
- this.sqlSchemaManager = sqlSchemaManager;
+ sqlSchemaManager = new CatalogSqlSchemaManager(catalogManager, 1000);
var executionSrvc = registerService(ExecutionServiceImpl.create(
clusterSrvc.topologyService(),
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistry.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistry.java
index 36b8cb4eb6..6461cae939 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistry.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistry.java
@@ -28,9 +28,10 @@ public interface ExecutableTableRegistry {
/**
* Returns an table that can be used for both read and write operations.
*
+ * @param schemaVersion Table schema version.
* @param tableId Table Id.
* @param tableDescriptor Table descriptor.
* @return An operation that returns executable table.
*/
- CompletableFuture<ExecutableTable> getTable(int tableId, TableDescriptor tableDescriptor);
+ CompletableFuture<ExecutableTable> getTable(int schemaVersion, int tableId, TableDescriptor tableDescriptor);
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
index 92b254855d..67c9b5c4fd 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
@@ -61,8 +61,10 @@ public class ExecutableTableRegistryImpl implements ExecutableTableRegistry, Sch
/** {@inheritDoc} */
@Override
- public CompletableFuture<ExecutableTable> getTable(int tableId, TableDescriptor tableDescriptor) {
- return tableCache.computeIfAbsent(tableId, (k) -> loadTable(k, tableDescriptor));
+ public CompletableFuture<ExecutableTable> getTable(int schemaVersion, int tableId, TableDescriptor tableDescriptor) {
+ //TODO: SchemaVersion must a part of cache key, or cache InternalTable only.
+ // return tableCache.computeIfAbsent(tableId, (k) -> loadTable(schemaVersion, k, tableDescriptor));
+ return loadTable(schemaVersion, tableId, tableDescriptor);
}
/** {@inheritDoc} */
@@ -71,27 +73,21 @@ public class ExecutableTableRegistryImpl implements ExecutableTableRegistry, Sch
tableCache.clear();
}
- private CompletableFuture<ExecutableTable> loadTable(int tableId, TableDescriptor tableDescriptor) {
-
- CompletableFuture<Map.Entry<InternalTable, SchemaRegistry>> f = tableManager.tableAsync(tableId)
+ private CompletableFuture<ExecutableTable> loadTable(int schemaVersion, int tableId, TableDescriptor tableDescriptor) {
+ return tableManager.tableAsync(tableId)
.thenApply(table -> {
InternalTable internalTable = table.internalTable();
SchemaRegistry schemaRegistry = schemaManager.schemaRegistry(tableId);
- return Map.entry(internalTable, schemaRegistry);
- });
- return f.thenApply((table) -> {
- SchemaRegistry schemaRegistry = table.getValue();
- SchemaDescriptor schemaDescriptor = schemaRegistry.schema();
- TableRowConverter rowConverter = new TableRowConverterImpl(schemaRegistry, schemaDescriptor, tableDescriptor);
- InternalTable internalTable = table.getKey();
- ScannableTable scannableTable = new ScannableTableImpl(internalTable, rowConverter, tableDescriptor);
+ SchemaDescriptor schemaDescriptor = schemaRegistry.schema(schemaVersion);
+ TableRowConverter rowConverter = new TableRowConverterImpl(schemaRegistry, schemaDescriptor, tableDescriptor);
+ ScannableTable scannableTable = new ScannableTableImpl(internalTable, rowConverter, tableDescriptor);
- UpdatableTableImpl updatableTable = new UpdatableTableImpl(tableId, tableDescriptor, internalTable.partitions(),
- replicaService, clock, rowConverter, schemaDescriptor);
+ UpdatableTableImpl updatableTable = new UpdatableTableImpl(tableId, tableDescriptor, internalTable.partitions(),
+ replicaService, clock, rowConverter, schemaDescriptor);
- return new ExecutableTableImpl(scannableTable, updatableTable);
- });
+ return new ExecutableTableImpl(scannableTable, updatableTable);
+ });
}
private static final class ExecutableTableImpl implements ExecutableTable {
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverImpl.java
index 4cd0b8c0ba..c07bdcc3f5 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverImpl.java
@@ -82,7 +82,7 @@ public class ExecutionDependencyResolverImpl implements ExecutionDependencyResol
int tableId = igniteTable.id();
TableDescriptor tableDescriptor = igniteTable.descriptor();
- tableMap.computeIfAbsent(tableId, (id) -> registry.getTable(tableId, tableDescriptor));
+ tableMap.computeIfAbsent(tableId, (id) -> registry.getTable((int) schemaVersion, tableId, tableDescriptor));
}
};
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
index d1bfb1187f..0a0f973698 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
@@ -389,18 +389,6 @@ public class LogicalRelImplementor<RowT> implements IgniteRelVisitor<Node<RowT>>
);
}
- private static RelDataType getIndexRowType(IgniteTypeFactory typeFactory, IgniteTable tbl, IgniteIndex idx) {
- RelDataTypeFactory.Builder b = new RelDataTypeFactory.Builder(typeFactory);
-
- RelDataType rt = tbl.getRowType(typeFactory);
- for (RelFieldCollation fc : idx.collation().getFieldCollations()) {
- b.add(rt.getFieldList().get(fc.getFieldIndex()));
- }
-
- RelDataType idxRowType = b.build();
- return idxRowType;
- }
-
/** {@inheritDoc} */
@Override
public Node<RowT> visit(IgniteTableScan rel) {
@@ -809,4 +797,16 @@ public class LogicalRelImplementor<RowT> implements IgniteRelVisitor<Node<RowT>>
public <T extends Node<RowT>> T go(IgniteRel rel) {
return (T) visit(rel);
}
+
+ private static RelDataType getIndexRowType(IgniteTypeFactory typeFactory, IgniteTable tbl, IgniteIndex idx) {
+ RelDataTypeFactory.Builder b = new RelDataTypeFactory.Builder(typeFactory);
+
+ RelDataType rt = tbl.getRowType(typeFactory);
+ for (RelFieldCollation fc : idx.collation().getFieldCollations()) {
+ b.add(rt.getFieldList().get(fc.getFieldIndex()));
+ }
+
+ RelDataType idxRowType = b.build();
+ return idxRowType;
+ }
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/CacheKey.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/CacheKey.java
index 89cd57d3a2..9decab5523 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/CacheKey.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/CacheKey.java
@@ -28,6 +28,8 @@ import java.util.Objects;
public class CacheKey {
public static final Class[] EMPTY_CLASS_ARRAY = {};
+ private final int version;
+
private final String schemaName;
private final String query;
@@ -39,29 +41,21 @@ public class CacheKey {
/**
* Constructor.
*
+ * @param version Catalog version.
* @param schemaName Schema name.
- * @param query Query string.
- * @param contextKey Optional context key to differ queries with and without/different flags, having an impact on result plan (like
- * LOCAL flag)
+ * @param query Query string.
+ * @param contextKey Optional context key to differ queries with and without/different flags, having an impact on result plan
+ * (like LOCAL flag)
* @param paramTypes Types of all dynamic parameters, no any type can be {@code null}.
*/
- public CacheKey(String schemaName, String query, Object contextKey, Class[] paramTypes) {
+ public CacheKey(int version, String schemaName, String query, Object contextKey, Class[] paramTypes) {
+ this.version = version;
this.schemaName = schemaName;
this.query = query;
this.contextKey = contextKey;
this.paramTypes = paramTypes;
}
- /**
- * Constructor.
- *
- * @param schemaName Schema name.
- * @param query Query string.
- */
- public CacheKey(String schemaName, String query) {
- this(schemaName, query, null, EMPTY_CLASS_ARRAY);
- }
-
/** {@inheritDoc} */
@Override
public boolean equals(Object o) {
@@ -74,6 +68,9 @@ public class CacheKey {
CacheKey cacheKey = (CacheKey) o;
+ if (version != cacheKey.version) {
+ return false;
+ }
if (!schemaName.equals(cacheKey.schemaName)) {
return false;
}
@@ -89,7 +86,8 @@ public class CacheKey {
@Override
public int hashCode() {
- int result = schemaName.hashCode();
+ int result = version;
+ result = 31 * result + schemaName.hashCode();
result = 31 * result + query.hashCode();
result = 31 * result + (contextKey != null ? contextKey.hashCode() : 0);
result = 31 * result + Arrays.deepHashCode(paramTypes);
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/PrepareServiceImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/PrepareServiceImpl.java
index cf04e7a29d..9b501ed331 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/PrepareServiceImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/PrepareServiceImpl.java
@@ -298,7 +298,9 @@ public class PrepareServiceImpl implements PrepareService, SchemaUpdateListener
? EMPTY_CLASS_ARRAY :
Arrays.stream(ctx.parameters()).map(p -> (p != null) ? p.getClass() : Void.class).toArray(Class[]::new);
- return new CacheKey(ctx.schemaName(), parsedResult.normalizedQuery(), distributed, paramTypes);
+ int version = (int) ctx.unwrap(BaseQueryContext.class).schemaVersion();
+
+ return new CacheKey(version, ctx.schemaName(), parsedResult.normalizedQuery(), distributed, paramTypes);
}
private ResultSetMetadata resultSetMetadata(
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistrySelfTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistrySelfTest.java
index c98ee0c454..740a1cba71 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistrySelfTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistrySelfTest.java
@@ -150,7 +150,7 @@ public class ExecutableTableRegistrySelfTest {
when(schemaManager.schemaRegistry(tableId)).thenReturn(schemaRegistry);
when(schemaRegistry.schema()).thenReturn(schemaDescriptor);
- return registry.getTable(tableId, descriptor);
+ return registry.getTable(schemaVersion, tableId, descriptor);
}
}
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
index 6a73dff178..75c1d14f66 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
@@ -98,8 +98,8 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
TableDescriptor td1 = tester.tableDescriptor("TEST1");
TableDescriptor td2 = tester.tableDescriptor("TEST2");
- verify(registry, times(1)).getTable(eq(t1Id), same(td1));
- verify(registry, times(1)).getTable(eq(t2Id), same(td2));
+ verify(registry, times(1)).getTable(-1, eq(t1Id), same(td1));
+ verify(registry, times(1)).getTable(-1, eq(t2Id), same(td2));
}
/**
@@ -124,7 +124,7 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
TableDescriptor td1 = tester.tableDescriptor("TEST1");
- verify(registry, times(1)).getTable(eq(t1Id), same(td1));
+ verify(registry, times(1)).getTable(-1, eq(t1Id), same(td1));
}
/**
@@ -155,8 +155,8 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
TableDescriptor td1 = tester.tableDescriptor("TEST1");
TableDescriptor td2 = tester.tableDescriptor("TEST2");
- verify(registry, times(1)).getTable(eq(t1Id), same(td1));
- verify(registry, times(1)).getTable(eq(t2Id), same(td2));
+ verify(registry, times(1)).getTable(-1, eq(t1Id), same(td1));
+ verify(registry, times(1)).getTable(-1, eq(t2Id), same(td2));
}
/**
@@ -179,7 +179,7 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
ResolvedDependencies deps = f.join();
tester.checkDependencies(deps, t1Id);
- verify(registry, times(1)).getTable(anyInt(), any(TableDescriptor.class));
+ verify(registry, times(1)).getTable(-1, anyInt(), any(TableDescriptor.class));
}
/**
@@ -229,14 +229,14 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
CompletableFuture<ExecutableTable> f = CompletableFuture.completedFuture(executableTable);
- when(registry.getTable(eq(tableId), any(TableDescriptor.class))).thenReturn(f);
+ when(registry.getTable(-1, eq(tableId), any(TableDescriptor.class))).thenReturn(f);
}
void setError(int tableId, Throwable err) {
CompletableFuture<ExecutableTable> f = new CompletableFuture<>();
f.completeExceptionally(err);
- when(registry.getTable(eq(tableId), any(TableDescriptor.class))).thenReturn(f);
+ when(registry.getTable(-1, eq(tableId), any(TableDescriptor.class))).thenReturn(f);
}
CompletableFuture<ResolvedDependencies> resolveDependencies(String sql) {
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/NoOpExecutableTableRegistry.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/NoOpExecutableTableRegistry.java
index 0acba776ca..f672fd14b5 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/NoOpExecutableTableRegistry.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/NoOpExecutableTableRegistry.java
@@ -25,7 +25,7 @@ public final class NoOpExecutableTableRegistry implements ExecutableTableRegistr
/** {@inheritDoc} */
@Override
- public CompletableFuture<ExecutableTable> getTable(int tableId, TableDescriptor tableDescriptor) {
+ public CompletableFuture<ExecutableTable> getTable(int schemaVersion, int tableId, TableDescriptor tableDescriptor) {
return CompletableFuture.completedFuture(new NoOpExecutableTable(tableId));
}
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index be426e99f6..a6dd0dd47e 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -1236,14 +1236,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param causalityToken Causality token.
* @param tableDescriptor Catalog table descriptor.
* @param zoneDescriptor Catalog distributed zone descriptor.
- * @param indexesToWait
+ * @param tableIndexes Ids of indexes belongs to the table.
* @return Future that will be completed when local changes related to the table creation are applied.
*/
private CompletableFuture<?> createTableLocally(
long causalityToken,
CatalogTableDescriptor tableDescriptor,
CatalogZoneDescriptor zoneDescriptor,
- int[] indexesToWait,
+ int[] tableIndexes,
List<Set<Assignment>> assignments
) {
String tableName = tableDescriptor.name();
@@ -1264,7 +1264,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
var table = new TableImpl(internalTable, lockMgr);
// TODO: IGNITE-19082 Need another way to wait for indexes
-// table.addIndexesToWait(indexesToWait);
+ table.addIndexesToWait(tableIndexes);
tablesByIdVv.update(causalityToken, (previous, e) -> inBusyLock(busyLock, () -> {
if (e != null) {
@@ -1292,8 +1292,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
pendingTables.remove(tableId);
}));
- createPartsFut
- .thenRunAsync(() -> inBusyLock(busyLock, () -> completeApiCreateFuture(table)));
+ createPartsFut.thenRunAsync(() -> inBusyLock(busyLock, () -> completeApiCreateFuture(table)));
// TODO should be reworked in IGNITE-16763
// We use the event notification future as the result so that dependent components can complete the schema updates.
[ignite-3] 09/19: Fix dependency resolving in SQL.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit e1d5c80761793484f81f339ea85445eece22de52
Author: amashenkov <an...@gmail.com>
AuthorDate: Wed Jul 19 19:08:58 2023 +0300
Fix dependency resolving in SQL.
---
.../apache/ignite/internal/index/IndexManager.java | 2 +-
.../apache/ignite/internal/schema/SchemaManager.java | 6 +++---
.../ignite/internal/sql/engine/SqlQueryProcessor.java | 1 -
.../sql/engine/exec/ExecutionServiceImpl.java | 19 ++++++++++++++-----
.../sql/engine/prepare/MappingQueryContext.java | 10 +++++++++-
.../internal/sql/engine/schema/IgniteSchemaTable.java | 2 +-
.../sql/engine/exec/ExecutionServiceImplTest.java | 3 ++-
.../internal/sql/engine/framework/TestNode.java | 4 ++--
.../internal/sql/engine/planner/PlannerTest.java | 2 +-
.../internal/table/distributed/TableManager.java | 9 ++++-----
10 files changed, 37 insertions(+), 21 deletions(-)
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index 555e05de18..89fee4b074 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -308,7 +308,7 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
});
- return createIndexFuture;
+ return createIndexFuture;
}
/**
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index 18d7775a0c..85edc90e35 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -121,7 +121,7 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
private CompletableFuture<?> onSchemaChange(CreateTableEventParameters evt) {
CatalogTableDescriptor tableDescriptor = evt.tableDescriptor();
- int newSchemaVersion = INITIAL_SCHEMA_VERSION;// evt.catalogVersion();
+ int newSchemaVersion = INITIAL_SCHEMA_VERSION; // evt.catalogVersion();
int tblId = tableDescriptor.id();
SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
@@ -196,9 +196,9 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
);
}
- return saveSchemaDescriptor(tblId, newSchema)
+ return saveSchemaDescriptor(tblId, newSchema)
.thenApply(t -> registerSchema(tblId, newSchema, registries));
- }));
+ }));
} finally {
busyLock.leaveBusy();
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index 010e7b1e4d..6d5cd4c58a 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -40,7 +40,6 @@ import org.apache.calcite.schema.SchemaPlus;
import org.apache.calcite.tools.Frameworks;
import org.apache.calcite.util.Pair;
import org.apache.ignite.internal.catalog.CatalogManager;
-import org.apache.ignite.internal.catalog.descriptors.CatalogSchemaDescriptor;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.index.IndexManager;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
index ea12b0a4e0..3cf9154624 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
@@ -85,6 +85,7 @@ import org.apache.ignite.internal.sql.engine.util.BaseQueryContext;
import org.apache.ignite.internal.sql.engine.util.Commons;
import org.apache.ignite.internal.sql.engine.util.HashFunctionFactoryImpl;
import org.apache.ignite.internal.sql.engine.util.TypeUtils;
+import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.internal.tx.InternalTransaction;
import org.apache.ignite.internal.util.ExceptionUtils;
import org.apache.ignite.lang.IgniteBiTuple;
@@ -128,6 +129,8 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
private final ExecutionDependencyResolver dependencyResolver;
+ private final TableManager tableManager;
+
private final ImplementorFactory<RowT> implementorFactory;
private final Map<UUID, DistributedQueryManager> queryManagerMap = new ConcurrentHashMap<>();
@@ -135,6 +138,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
/**
* Creates the execution services.
*
+ * @param <RowT> Type of the sql row.
* @param topSrvc Topology service.
* @param msgSrvc Message service.
* @param sqlSchemaManager Schema manager.
@@ -143,7 +147,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
* @param handler Row handler.
* @param mailboxRegistry Mailbox registry.
* @param exchangeSrvc Exchange service.
- * @param <RowT> Type of the sql row.
+ * @param tableManager Table Manager.
* @return An execution service.
*/
public static <RowT> ExecutionServiceImpl<RowT> create(
@@ -155,7 +159,8 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
RowHandler<RowT> handler,
MailboxRegistry mailboxRegistry,
ExchangeService exchangeSrvc,
- ExecutionDependencyResolver dependencyResolver
+ ExecutionDependencyResolver dependencyResolver,
+ TableManager tableManager
) {
return new ExecutionServiceImpl<>(
msgSrvc,
@@ -171,7 +176,8 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
new HashFunctionFactoryImpl<>(sqlSchemaManager, handler),
mailboxRegistry,
exchangeSrvc,
- deps)
+ deps),
+ tableManager
);
}
@@ -186,6 +192,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
* @param taskExecutor Task executor.
* @param handler Row handler.
* @param implementorFactory Relational node implementor factory.
+ * @param tableManager Table manager.
*/
public ExecutionServiceImpl(
MessageService messageService,
@@ -196,7 +203,8 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
QueryTaskExecutor taskExecutor,
RowHandler<RowT> handler,
ExecutionDependencyResolver dependencyResolver,
- ImplementorFactory<RowT> implementorFactory
+ ImplementorFactory<RowT> implementorFactory,
+ TableManager tableManager
) {
this.localNode = topSrvc.localMember();
this.handler = handler;
@@ -208,6 +216,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
this.ddlCmdHnd = ddlCmdHnd;
this.dependencyResolver = dependencyResolver;
this.implementorFactory = implementorFactory;
+ this.tableManager = tableManager;
}
/** {@inheritDoc} */
@@ -639,7 +648,7 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
private AsyncCursor<List<Object>> execute(InternalTransaction tx, MultiStepPlan plan) {
taskExecutor.execute(() -> {
try {
- plan.init(new MappingQueryContext(localNode.name(), mappingSrvc));
+ plan.init(new MappingQueryContext(localNode.name(), mappingSrvc, tableManager));
List<Fragment> fragments = plan.fragments();
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/MappingQueryContext.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/MappingQueryContext.java
index d1ebb46fad..da7658b596 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/MappingQueryContext.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/prepare/MappingQueryContext.java
@@ -23,6 +23,7 @@ import org.apache.ignite.internal.sql.engine.metadata.IgniteMetadata;
import org.apache.ignite.internal.sql.engine.metadata.MappingService;
import org.apache.ignite.internal.sql.engine.metadata.RelMetadataQueryEx;
import org.apache.ignite.internal.sql.engine.util.Commons;
+import org.apache.ignite.internal.table.distributed.TableManager;
/**
* Query mapping context.
@@ -30,6 +31,7 @@ import org.apache.ignite.internal.sql.engine.util.Commons;
public class MappingQueryContext {
private final String locNodeName;
private final MappingService mappingService;
+ private final TableManager tableManager;
private RelOptCluster cluster;
@@ -41,10 +43,12 @@ public class MappingQueryContext {
*/
public MappingQueryContext(
String locNodeName,
- MappingService mappingService
+ MappingService mappingService,
+ TableManager tableManager
) {
this.locNodeName = locNodeName;
this.mappingService = mappingService;
+ this.tableManager = tableManager;
}
/** Creates a cluster. */
@@ -66,4 +70,8 @@ public class MappingQueryContext {
public MappingService mappingService() {
return mappingService;
}
+
+ public TableManager tableManager() {
+ return tableManager;
+ }
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
index 4181054e0d..e6d34557e7 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
@@ -193,7 +193,7 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
/** {@inheritDoc} */
@Override
public ColocationGroup colocationGroup(MappingQueryContext ctx) {
- throw new UnsupportedOperationException("Execution related methods are not available.");
+ return IgniteTableImpl.partitionedGroup(ctx.tableManager().getTable(id).internalTable()).get();
}
/** {@inheritDoc} */
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImplTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImplTest.java
index d865f11f61..bbe6f06d2b 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImplTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImplTest.java
@@ -613,7 +613,8 @@ public class ExecutionServiceImplTest {
taskExecutor,
ArrayRowHandler.INSTANCE,
dependencyResolver,
- (ctx, deps) -> node.implementor(ctx, mailboxRegistry, exchangeService, deps)
+ (ctx, deps) -> node.implementor(ctx, mailboxRegistry, exchangeService, deps),
+ null
);
taskExecutor.start();
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestNode.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestNode.java
index 934d30d4c7..632ee782a8 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestNode.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestNode.java
@@ -150,8 +150,8 @@ public class TestNode implements LifecycleAware {
return new ScanNode<>(ctx, dataProvider);
}
- }
- ));
+ },
+ null));
parserService = new ParserServiceImpl(0, EmptyCacheFactory.INSTANCE);
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
index 2f887ffb83..3499c20e0b 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
@@ -915,6 +915,6 @@ public class PlannerTest extends AbstractPlannerTest {
private static MappingQueryContext mapContext(String locNodeName,
MappingService mappingService) {
- return new MappingQueryContext(locNodeName, mappingService);
+ return new MappingQueryContext(locNodeName, mappingService, null);
}
}
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 961486158b..655c584dff 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -202,7 +202,6 @@ import org.apache.ignite.lang.DistributionZoneNotFoundException;
import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteExceptionUtils;
import org.apache.ignite.lang.IgniteInternalException;
-import org.apache.ignite.lang.IgniteStringFormatter;
import org.apache.ignite.lang.IgniteSystemProperties;
import org.apache.ignite.lang.NodeStoppingException;
import org.apache.ignite.lang.TableAlreadyExistsException;
@@ -1292,10 +1291,9 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
// We use the event notification future as the result so that dependent components can complete the schema updates.
// TODO: https://issues.apache.org/jira/browse/IGNITE-19913 Possible performance degradation.
- CompletableFuture<?> eventFut = fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
+ fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
- // TODO: investigate why createParts and eventFutures hangs.
-// return allOf(createPartsFut, eventFut);
+ // TODO: investigate why createParts hangs.
return completedFuture(false);
}
@@ -2851,7 +2849,8 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
if (primaryCols.contains(colName)) {
- throw new SqlException(STMT_VALIDATION_ERR, format("Can`t delete column, belongs to primary key: [name={}]", colName));
+ throw new SqlException(STMT_VALIDATION_ERR,
+ format("Can`t delete column, belongs to primary key: [name={}]", colName));
}
}
[ignite-3] 13/19: Fix column index mapping.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 0f46f6fdde74fcf9d93567de146e860f4d787017
Author: amashenkov <an...@gmail.com>
AuthorDate: Fri Jul 21 02:24:16 2023 +0300
Fix column index mapping.
---
.../sql/engine/exec/TableRowConverterImpl.java | 22 ++++++++++++----------
.../sql/engine/exec/UpdatableTableImpl.java | 22 ++++++++++++++--------
.../sql/engine/schema/IgniteSchemaTable.java | 4 ++--
3 files changed, 28 insertions(+), 20 deletions(-)
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/TableRowConverterImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/TableRowConverterImpl.java
index dc81c12c76..f4f7f1d254 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/TableRowConverterImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/TableRowConverterImpl.java
@@ -36,13 +36,19 @@ public class TableRowConverterImpl implements TableRowConverter {
private final SchemaDescriptor schemaDescriptor;
- private final TableDescriptor desc;
+ private final int[] physicalIndexMap;
/** Constructor. */
public TableRowConverterImpl(SchemaRegistry schemaRegistry, SchemaDescriptor schemaDescriptor, TableDescriptor desc) {
this.schemaRegistry = schemaRegistry;
this.schemaDescriptor = schemaDescriptor;
- this.desc = desc;
+
+ physicalIndexMap = new int[desc.columnsCount()];
+
+ for (int i = 0; i < desc.columnsCount(); i++) {
+ ColumnDescriptor col = desc.columnDescriptor(i);
+ physicalIndexMap[i] = schemaDescriptor.column(col.name()).schemaIndex();
+ }
}
/** {@inheritDoc} */
@@ -59,21 +65,17 @@ public class TableRowConverterImpl implements TableRowConverter {
RowT res = factory.create();
- assert handler.columnCount(res) == (requiredColumns == null ? desc.columnsCount() : requiredColumns.cardinality());
+ assert handler.columnCount(res) == (requiredColumns == null ? physicalIndexMap.length : requiredColumns.cardinality());
Row row = schemaRegistry.resolve(binaryRow, schemaDescriptor);
if (requiredColumns == null) {
- for (int i = 0; i < desc.columnsCount(); i++) {
- ColumnDescriptor colDesc = desc.columnDescriptor(i);
-
- handler.set(i, res, TypeUtils.toInternal(row.value(colDesc.physicalIndex())));
+ for (int i = 0; i < physicalIndexMap.length; i++) {
+ handler.set(i, res, TypeUtils.toInternal(row.value(physicalIndexMap[i])));
}
} else {
for (int i = 0, j = requiredColumns.nextSetBit(0); j != -1; j = requiredColumns.nextSetBit(j + 1), i++) {
- ColumnDescriptor colDesc = desc.columnDescriptor(j);
-
- handler.set(i, res, TypeUtils.toInternal(row.value(colDesc.physicalIndex())));
+ handler.set(i, res, TypeUtils.toInternal(row.value(physicalIndexMap[j])));
}
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
index 310423aca4..3f48868249 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
@@ -43,6 +43,7 @@ import org.apache.ignite.internal.schema.row.Row;
import org.apache.ignite.internal.schema.row.RowAssembler;
import org.apache.ignite.internal.sql.engine.metadata.NodeWithTerm;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
+import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptorImpl;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.TypeUtils;
@@ -69,7 +70,7 @@ public final class UpdatableTableImpl implements UpdatableTable {
private final SchemaDescriptor schemaDescriptor;
- private final List<ColumnDescriptor> columnsOrderedByPhysSchema;
+ private final int[] columnsOrderedByPhysSchema;
private final PartitionExtractor partitionExtractor;
@@ -96,14 +97,15 @@ public final class UpdatableTableImpl implements UpdatableTable {
this.schemaDescriptor = schemaDescriptor;
this.partitionExtractor = (row) -> IgniteUtils.safeAbs(row.colocationHash()) % partitions;
- List<ColumnDescriptor> tmp = new ArrayList<>(desc.columnsCount());
+ columnsOrderedByPhysSchema = new int[desc.columnsCount()];
for (int i = 0; i < desc.columnsCount(); i++) {
- tmp.add(desc.columnDescriptor(i));
- }
+ ColumnDescriptor col = desc.columnDescriptor(i);
+
+ int physIndex = schemaDescriptor.column(col.name()).schemaIndex();
- tmp.sort(Comparator.comparingInt(ColumnDescriptor::physicalIndex));
+ columnsOrderedByPhysSchema[physIndex] = col.logicalIndex();
+ }
- columnsOrderedByPhysSchema = tmp;
this.rowConverter = rowConverter;
}
@@ -276,7 +278,9 @@ public final class UpdatableTableImpl implements UpdatableTable {
private <RowT> BinaryRowEx convertRow(RowT row, ExecutionContext<RowT> ectx, boolean keyOnly) {
RowHandler<RowT> hnd = ectx.rowHandler();
- for (ColumnDescriptor colDesc : columnsOrderedByPhysSchema) {
+ for (int i : columnsOrderedByPhysSchema) {
+ ColumnDescriptor colDesc = desc.columnDescriptor(i);
+
if (keyOnly && !colDesc.key()) {
continue;
}
@@ -293,7 +297,9 @@ public final class UpdatableTableImpl implements UpdatableTable {
RowAssembler rowAssembler = keyOnly ? RowAssembler.keyAssembler(schemaDescriptor) : new RowAssembler(schemaDescriptor);
- for (ColumnDescriptor colDesc : columnsOrderedByPhysSchema) {
+ for (int i : columnsOrderedByPhysSchema) {
+ ColumnDescriptor colDesc = desc.columnDescriptor(i);
+
if (keyOnly && !colDesc.key()) {
continue;
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
index 367953d654..85e1cb46da 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
@@ -200,8 +200,8 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
/** {@inheritDoc} */
@Override
public <C> @Nullable C unwrap(Class<C> cls) {
- if (cls.isInstance(this)) {
- return cls.cast(this);
+ if (cls.isInstance(desc)) {
+ return cls.cast(desc);
}
return super.unwrap(cls);
[ignite-3] 08/19: Switch other managers to Catalog events.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit e1f9fa90797780734bbae25d15ef6b28269006d9
Author: amashenkov <an...@gmail.com>
AuthorDate: Thu Jul 13 13:23:23 2023 +0300
Switch other managers to Catalog events.
---
.../ignite/internal/catalog/CatalogService.java | 1 +
.../apache/ignite/internal/index/IndexManager.java | 314 +++------------------
.../ignite/internal/index/IndexManagerTest.java | 2 +-
.../storage/ItRebalanceDistributedTest.java | 2 +-
.../runner/app/ItIgniteNodeRestartTest.java | 4 +-
.../org/apache/ignite/internal/app/IgniteImpl.java | 6 +-
.../internal/schema/CatalogDescriptorUtils.java | 50 ++++
.../ignite/internal/schema/SchemaManager.java | 110 +++++---
.../internal/sql/engine/SqlQueryProcessor.java | 34 +--
.../sql/engine/exec/ExecutionServiceImpl.java | 7 +-
.../sql/engine/schema/CatalogSqlSchemaManager.java | 11 +-
.../internal/sql/engine/util/BaseQueryContext.java | 4 +-
.../sql/engine/exec/MockedStructuresTest.java | 4 +-
.../internal/table/distributed/TableManager.java | 71 ++---
.../table/distributed/TableManagerTest.java | 11 +-
15 files changed, 235 insertions(+), 396 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
index 44ab7ed8de..fd710f8b1f 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogService.java
@@ -63,6 +63,7 @@ public interface CatalogService {
CatalogZoneDescriptor zone(String zoneName, long timestamp);
+ @Deprecated
CatalogZoneDescriptor zone(int zoneId, long timestamp);
CatalogZoneDescriptor zone(int zoneId, int version);
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index ff34697c24..555e05de18 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -18,23 +18,15 @@
package org.apache.ignite.internal.index;
import static java.util.concurrent.CompletableFuture.allOf;
-import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
-import static org.apache.ignite.internal.schema.CatalogDescriptorUtils.toIndexDescriptor;
-import static org.apache.ignite.internal.schema.CatalogDescriptorUtils.toTableDescriptor;
-import static org.apache.ignite.internal.schema.configuration.SchemaConfigurationUtils.findTableView;
import static org.apache.ignite.internal.util.ArrayUtils.STRING_EMPTY_ARRAY;
import java.util.ArrayList;
-import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicBoolean;
-import java.util.function.Consumer;
import java.util.function.Function;
-import org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
-import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.catalog.commands.AbstractIndexCommandParams;
import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
@@ -46,6 +38,9 @@ import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescript
import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogSortedIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CreateIndexEventParameters;
+import org.apache.ignite.internal.catalog.events.DropIndexEventParameters;
import org.apache.ignite.internal.index.event.IndexEvent;
import org.apache.ignite.internal.index.event.IndexEventParameters;
import org.apache.ignite.internal.logger.IgniteLogger;
@@ -59,31 +54,18 @@ import org.apache.ignite.internal.schema.Column;
import org.apache.ignite.internal.schema.SchemaDescriptor;
import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.SchemaRegistry;
-import org.apache.ignite.internal.schema.configuration.TableView;
-import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
-import org.apache.ignite.internal.schema.configuration.TablesView;
-import org.apache.ignite.internal.schema.configuration.index.HashIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.SortedIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexConfiguration;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
import org.apache.ignite.internal.storage.index.StorageHashIndexDescriptor;
import org.apache.ignite.internal.storage.index.StorageIndexDescriptor;
import org.apache.ignite.internal.storage.index.StorageSortedIndexDescriptor;
import org.apache.ignite.internal.table.TableImpl;
import org.apache.ignite.internal.table.distributed.PartitionSet;
import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.internal.table.event.TableEvent;
import org.apache.ignite.internal.util.IgniteSpinBusyLock;
import org.apache.ignite.internal.util.StringUtils;
import org.apache.ignite.lang.ErrorGroups;
-import org.apache.ignite.lang.ErrorGroups.Common;
import org.apache.ignite.lang.IgniteException;
import org.apache.ignite.lang.IgniteInternalException;
-import org.apache.ignite.lang.IndexAlreadyExistsException;
-import org.apache.ignite.lang.IndexNotFoundException;
import org.apache.ignite.lang.NodeStoppingException;
-import org.apache.ignite.lang.TableNotFoundException;
/**
* An Ignite component that is responsible for handling index-related commands like CREATE or DROP
@@ -93,9 +75,6 @@ import org.apache.ignite.lang.TableNotFoundException;
public class IndexManager extends Producer<IndexEvent, IndexEventParameters> implements IgniteComponent {
private static final IgniteLogger LOG = Loggers.forClass(IndexManager.class);
- /** Common tables and indexes configuration. */
- private final TablesConfiguration tablesCfg;
-
/** Schema manager. */
private final SchemaManager schemaManager;
@@ -114,18 +93,15 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
/**
* Constructor.
*
- * @param tablesCfg Tables and indexes configuration.
* @param catalogManager Catalog manager.
* @param schemaManager Schema manager.
* @param tableManager Table manager.
*/
public IndexManager(
- TablesConfiguration tablesCfg,
CatalogManager catalogManager,
SchemaManager schemaManager,
TableManager tableManager
) {
- this.tablesCfg = Objects.requireNonNull(tablesCfg, "tablesCfg");
this.schemaManager = Objects.requireNonNull(schemaManager, "schemaManager");
this.tableManager = tableManager;
this.catalogManager = catalogManager;
@@ -136,40 +112,10 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
public void start() {
LOG.debug("Index manager is about to start");
- tablesCfg.indexes().listenElements(new ConfigurationListener());
-
- tableManager.listen(TableEvent.CREATE, (param, ex) -> {
- if (ex != null) {
- return completedFuture(false);
- }
-
- // We can't return this future as the listener's result, because a deadlock can happen in the configuration component:
- // this listener is called inside a configuration notification thread and all notifications are required to finish before
- // new configuration modifications can occur (i.e. we are creating an index below). Therefore we create the index fully
- // asynchronously and rely on the underlying components to handle PK index synchronisation.
- tableManager.tableAsync(param.causalityToken(), param.tableId())
- .thenCompose(table -> {
- String[] pkColumns = Arrays.stream(table.schemaView().schema().keyColumns().columns())
- .map(Column::name)
- .toArray(String[]::new);
-
- String pkName = table.name() + "_PK";
- int pkId = table.tableId() + 1;
-
- // Update config bypassing the Catalog, because PK has just been created in Catalog with the table.
- return createIndexInternal(pkId, "PUBLIC", pkName, table.name(), false,
- change -> change.changeUniq(true).convert(HashIndexChange.class)
- .changeColumnNames(pkColumns)
- );
- })
- .whenComplete((v, e) -> {
- if (e != null) {
- LOG.error("Error when creating index: " + e);
- }
- });
-
- return completedFuture(false);
- });
+ catalogManager.listen(CatalogEvent.INDEX_CREATE,
+ (params, ex) -> onIndexCreate((CreateIndexEventParameters) params).thenApply(ignore -> false));
+ catalogManager.listen(CatalogEvent.INDEX_DROP,
+ (params, ex) -> onIndexDrop((DropIndexEventParameters) params).thenApply(ignore -> false));
LOG.info("Index manager started");
}
@@ -199,119 +145,19 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
try {
+ if (StringUtils.nullOrEmpty(params.indexName())) {
+ return failedFuture(new IgniteInternalException(
+ ErrorGroups.Index.INVALID_INDEX_DEFINITION_ERR,
+ "Index name should be at least 1 character long"
+ ));
+ }
+
CompletableFuture<Void> indexCreateFuture = (params instanceof CreateSortedIndexParams)
? catalogManager.createIndex((CreateSortedIndexParams) params)
: catalogManager.createIndex((CreateHashIndexParams) params);
return indexCreateFuture.thenApply(ignore -> catalogManager.index(params.indexName(), Long.MAX_VALUE))
- .thenCompose(index -> createIndexInternal(
- index.id(),
- params.schemaName(),
- params.indexName(),
- params.tableName(),
- true,
- createIndexChanger(params)
- ));
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- @Deprecated(forRemoval = true)
- private Consumer<TableIndexChange> createIndexChanger(AbstractIndexCommandParams indexParams) {
- if (indexParams instanceof CreateSortedIndexParams) {
- CreateSortedIndexParams params = (CreateSortedIndexParams) indexParams;
- return tableIndexChange -> tableIndexChange.convert(SortedIndexChange.class).changeColumns(colsInit -> {
- for (int i = 0; i < params.columns().size(); i++) {
- String columnName = params.columns().get(i);
- CatalogColumnCollation collation = params.collations().get(i);
- //TODO: https://issues.apache.org/jira/browse/IGNITE-17563 Pass null ordering for columns.
- colsInit.create(columnName, colInit -> colInit.changeAsc(collation.asc()));
- }
- });
- } else {
- CreateHashIndexParams params = (CreateHashIndexParams) indexParams;
- return tableIndexChange -> tableIndexChange.convert(HashIndexChange.class)
- .changeColumnNames(params.columns().toArray(STRING_EMPTY_ARRAY));
- }
- }
-
- @Deprecated(forRemoval = true)
- private CompletableFuture<Boolean> createIndexInternal(
- int indexId,
- String schemaName,
- String indexName,
- String tableName,
- boolean failIfExists,
- Consumer<TableIndexChange> indexChange
- ) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new NodeStoppingException());
- }
-
- LOG.debug("Going to create index [schema={}, table={}, index={}]", schemaName, tableName, indexName);
-
- try {
- validateName(indexName);
-
- CompletableFuture<Boolean> future = new CompletableFuture<>();
-
- // Check index existence flag, avoid usage of hasCause + IndexAlreadyExistsException.
- AtomicBoolean idxExist = new AtomicBoolean(false);
-
- tablesCfg.change(tablesChange -> tablesChange.changeIndexes(indexListChange -> {
- idxExist.set(false);
-
- if (indexListChange.get(indexName) != null) {
- idxExist.set(true);
-
- throw new IndexAlreadyExistsException(schemaName, indexName);
- }
-
- TableView tableCfg = tablesChange.tables().get(tableName);
-
- if (tableCfg == null) {
- throw new TableNotFoundException(schemaName, tableName);
- }
-
- int tableId = tableCfg.id();
-
- tablesChange.changeGlobalIdCounter(indexId);
-
- Consumer<TableIndexChange> chg = indexChange.andThen(c -> c.changeTableId(tableId).changeId(indexId));
-
- indexListChange.create(indexName, chg);
- })).whenComplete((index, th) -> {
- if (th != null) {
- LOG.debug("Unable to create index [schema={}, table={}, index={}]",
- th, schemaName, tableName, indexName);
-
- if (!failIfExists && idxExist.get()) {
- future.complete(false);
- } else {
- future.completeExceptionally(th);
- }
- } else {
- TableIndexConfiguration idxCfg = tablesCfg.indexes().get(indexName);
-
- if (idxCfg != null && idxCfg.value() != null) {
- LOG.info("Index created [schema={}, table={}, index={}, indexId={}]",
- schemaName, tableName, indexName, idxCfg.id().value());
-
- future.complete(true);
- } else {
- var exception = new IgniteInternalException(
- Common.INTERNAL_ERR, "Looks like the index was concurrently deleted");
-
- LOG.info("Unable to create index [schema={}, table={}, index={}]",
- exception, schemaName, tableName, indexName);
-
- future.completeExceptionally(exception);
- }
- }
- });
-
- return future;
+ .thenApply(ignore -> false);
} finally {
busyLock.leaveBusy();
}
@@ -326,89 +172,22 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
try {
return catalogManager.dropIndex(params)
- .thenCompose(ignore -> dropIndexAsyncInternal(params.schemaName(), params.indexName(), true));
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- @Deprecated(forRemoval = true)
- private CompletableFuture<Boolean> dropIndexAsyncInternal(
- String schemaName,
- String indexName,
- boolean failIfNotExists
- ) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new NodeStoppingException());
- }
-
- LOG.debug("Going to drop index [schema={}, index={}]", schemaName, indexName);
-
- try {
- validateName(indexName);
-
- final CompletableFuture<Boolean> future = new CompletableFuture<>();
-
- // Check index existence flag, avoid usage of hasCause + IndexAlreadyExistsException.
- AtomicBoolean idxOrTblNotExist = new AtomicBoolean(false);
-
- tablesCfg.indexes().change(indexListChange -> {
- idxOrTblNotExist.set(false);
-
- TableIndexView idxView = indexListChange.get(indexName);
-
- if (idxView == null) {
- idxOrTblNotExist.set(true);
-
- throw new IndexNotFoundException(schemaName, indexName);
- }
-
- indexListChange.delete(indexName);
- }).whenComplete((ignored, th) -> {
- if (th != null) {
- LOG.info("Unable to drop index [schema={}, index={}]", th, schemaName, indexName);
-
- if (!failIfNotExists && idxOrTblNotExist.get()) {
- future.complete(false);
- } else {
- future.completeExceptionally(th);
- }
- } else {
- LOG.info("Index dropped [schema={}, index={}]", schemaName, indexName);
-
- future.complete(true);
- }
- });
-
- return future;
+ .thenApply(ignore -> false);
} finally {
busyLock.leaveBusy();
}
}
- private void validateName(String indexName) {
- if (StringUtils.nullOrEmpty(indexName)) {
- throw new IgniteInternalException(
- ErrorGroups.Index.INVALID_INDEX_DEFINITION_ERR,
- "Index name should be at least 1 character long"
- );
- }
- }
-
/**
* Callback method is called when index configuration changed and an index was dropped.
*
* @param evt Index configuration event.
* @return A future.
*/
- private CompletableFuture<?> onIndexDrop(ConfigurationNotificationEvent<TableIndexView> evt) {
- TableIndexView tableIndexView = evt.oldValue();
-
- int idxId = tableIndexView.id();
-
- int tableId = tableIndexView.tableId();
-
- long causalityToken = evt.storageRevision();
+ private CompletableFuture<?> onIndexDrop(DropIndexEventParameters evt) {
+ int idxId = evt.indexId();
+ int tableId = evt.tableId();
+ long causalityToken = evt.causalityToken();
if (!busyLock.enterBusy()) {
fireEvent(IndexEvent.DROP,
@@ -422,14 +201,20 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
try {
CompletableFuture<?> fireEventFuture = fireEvent(IndexEvent.DROP, new IndexEventParameters(causalityToken, tableId, idxId));
- CompletableFuture<?> dropIndexFuture = tableManager.tableAsync(causalityToken, tableId)
+ catalogManager.table(tableId, evt.catalogVersion());
+ CompletableFuture<?> dropIndexFuture = tableManager.tableAsync(tableId)
.thenAccept(table -> {
if (table != null) { // in case of DROP TABLE the table will be removed first
table.unregisterIndex(idxId);
}
});
+ // TODO: investigate why DropIndexFuture hangs.
return allOf(fireEventFuture, dropIndexFuture);
+ } catch (Throwable th) {
+ LOG.warn("Failed to process drop index event.", th);
+
+ return failedFuture(th);
} finally {
busyLock.leaveBusy();
}
@@ -441,16 +226,12 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
* @param evt Index configuration changed event.
* @return A future.
*/
- private CompletableFuture<?> onIndexCreate(ConfigurationNotificationEvent<TableIndexView> evt) {
- TableIndexView indexConfig = evt.newValue();
-
- int tableId = indexConfig.tableId();
+ private CompletableFuture<?> onIndexCreate(CreateIndexEventParameters evt) {
+ CatalogIndexDescriptor indexDescriptor = evt.indexDescriptor();
if (!busyLock.enterBusy()) {
- int idxId = indexConfig.id();
-
fireEvent(IndexEvent.CREATE,
- new IndexEventParameters(evt.storageRevision(), tableId, idxId),
+ new IndexEventParameters(evt.causalityToken(), indexDescriptor.tableId(), indexDescriptor.id()),
new NodeStoppingException()
);
@@ -458,16 +239,15 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
try {
- TablesView tablesView = evt.newValue(TablesView.class);
-
- TableView tableView = findTableView(tablesView.tables(), tableId);
+ CatalogTableDescriptor tableDescriptor = catalogManager.table(indexDescriptor.tableId(), evt.catalogVersion());
- assert tableView != null : "tableId=" + tableId + ", indexId=" + indexConfig.id();
+ assert tableDescriptor != null : "Table not found: tableId=" + indexDescriptor.tableId();
- CatalogTableDescriptor tableDescriptor = toTableDescriptor(tableView);
- CatalogIndexDescriptor indexDescriptor = toIndexDescriptor(indexConfig);
+ return createIndexLocally(evt.causalityToken(), tableDescriptor, indexDescriptor);
+ } catch (Throwable th) {
+ LOG.warn("Failed to process create index event.", th);
- return createIndexLocally(evt.storageRevision(), tableDescriptor, indexDescriptor);
+ return failedFuture(th);
} finally {
busyLock.leaveBusy();
}
@@ -615,28 +395,6 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
}
- private class ConfigurationListener implements ConfigurationNamedListListener<TableIndexView> {
- @Override
- public CompletableFuture<?> onCreate(ConfigurationNotificationEvent<TableIndexView> ctx) {
- return onIndexCreate(ctx);
- }
-
- @Override
- public CompletableFuture<?> onRename(ConfigurationNotificationEvent<TableIndexView> ctx) {
- return failedFuture(new UnsupportedOperationException("https://issues.apache.org/jira/browse/IGNITE-16196"));
- }
-
- @Override
- public CompletableFuture<?> onDelete(ConfigurationNotificationEvent<TableIndexView> ctx) {
- return onIndexDrop(ctx);
- }
-
- @Override
- public CompletableFuture<?> onUpdate(ConfigurationNotificationEvent<TableIndexView> ctx) {
- return failedFuture(new IllegalStateException("Should not be called"));
- }
- }
-
/**
* Converts a catalog index descriptor to an event index descriptor.
*
diff --git a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
index 53b2abf6d2..f0e0cb527e 100644
--- a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
+++ b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
@@ -120,7 +120,7 @@ public class IndexManagerTest {
when(catalogManager.index(anyString(), anyLong())).thenReturn(indexDescriptor);
when(indexDescriptor.id()).thenReturn(1);
- indexManager = new IndexManager(tablesConfig, catalogManager, schManager, tableManagerMock);
+ indexManager = new IndexManager(catalogManager, schManager, tableManagerMock);
indexManager.start();
assertThat(
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
index dba0049506..b0bafa6b56 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
@@ -784,7 +784,7 @@ public class ItRebalanceDistributedTest {
clockWaiter
);
- schemaManager = new SchemaManager(registry, tablesCfg, metaStorageManager);
+ schemaManager = new SchemaManager(registry, metaStorageManager, catalogManager);
distributionZoneManager = new DistributionZoneManager(
zonesCfg,
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 60e2e37a26..11aed86bdf 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -346,7 +346,7 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
clockWaiter
);
- SchemaManager schemaManager = new SchemaManager(registry, tablesConfig, metaStorageMgr);
+ SchemaManager schemaManager = new SchemaManager(registry, metaStorageMgr, catalogManager);
DistributionZoneManager distributionZoneManager = new DistributionZoneManager(
zonesConfig,
@@ -386,7 +386,7 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
null
);
- var indexManager = new IndexManager(tablesConfig, catalogManager, schemaManager, tableManager);
+ var indexManager = new IndexManager(catalogManager, schemaManager, tableManager);
SqlQueryProcessor qryEngine = new SqlQueryProcessor(
registry,
diff --git a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
index a9cd3b548b..8c66e13188 100644
--- a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
+++ b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
@@ -496,8 +496,6 @@ public class IgniteImpl implements Ignite {
)
);
- schemaManager = new SchemaManager(registry, tablesConfig, metaStorageMgr);
-
SchemaSynchronizationConfiguration schemaSyncConfig = clusterConfigRegistry.getConfiguration(
SchemaSynchronizationConfiguration.KEY
);
@@ -508,6 +506,8 @@ public class IgniteImpl implements Ignite {
() -> schemaSyncConfig.delayDuration().value()
);
+ schemaManager = new SchemaManager(registry, metaStorageMgr, catalogManager);
+
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfig,
@@ -550,7 +550,7 @@ public class IgniteImpl implements Ignite {
distributionZoneManager
);
- indexManager = new IndexManager(tablesConfig, catalogManager, schemaManager, distributedTblMgr);
+ indexManager = new IndexManager(catalogManager, schemaManager, distributedTblMgr);
qryEngine = new SqlQueryProcessor(
registry,
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
index bcff576238..e400dd1d11 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
@@ -21,6 +21,7 @@ import static java.util.stream.Collectors.toList;
import static org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation.ASC_NULLS_LAST;
import static org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation.DESC_NULLS_FIRST;
+import java.util.ArrayList;
import java.util.List;
import org.apache.ignite.internal.catalog.commands.CatalogUtils;
import org.apache.ignite.internal.catalog.commands.ColumnParams;
@@ -309,4 +310,53 @@ public class CatalogDescriptorUtils {
throw new IllegalArgumentException("Unknown type " + colType.spec().name());
}
}
+
+ /**
+ * Converts table descriptor to table schema descriptor.
+ */
+ public static SchemaDescriptor convert(int schemaVer, CatalogTableDescriptor tableDescriptor) {
+ List<Column> keyCols = new ArrayList<>(tableDescriptor.primaryKeyColumns().size());
+ List<Column> valCols = new ArrayList<>(tableDescriptor.columns().size() - tableDescriptor.primaryKeyColumns().size());
+
+ int idx = 0;
+
+ for (CatalogTableColumnDescriptor column : tableDescriptor.columns()) {
+ if (tableDescriptor.isPrimaryKeyColumn(column.name())) {
+ keyCols.add(convert(idx, column));
+ } else {
+ valCols.add(convert(idx, column));
+ }
+
+ idx++;
+ }
+
+ return new SchemaDescriptor(
+ schemaVer,
+ keyCols.toArray(Column[]::new),
+ tableDescriptor.colocationColumns().toArray(String[]::new),
+ valCols.toArray(Column[]::new)
+ );
+ }
+
+ private static Column convert(int columnOrder, CatalogTableColumnDescriptor columnDescriptor) {
+ NativeType type = getNativeType(columnDescriptor);
+
+ DefaultValue defaultValue = columnDescriptor.defaultValue();
+
+ DefaultValueProvider defaultValueProvider = toValueProvider(defaultValue);
+
+ return new Column(columnOrder, columnDescriptor.name(), type, columnDescriptor.nullable(), defaultValueProvider);
+ }
+
+ private static DefaultValueProvider toValueProvider(DefaultValue defaultValue) {
+ switch (defaultValue.type()) {
+ case CONSTANT:
+ return DefaultValueProvider.constantProvider(((ConstantValue) defaultValue).value());
+ case FUNCTION_CALL:
+ return DefaultValueProvider.forValueGenerator(
+ DefaultValueGenerator.valueOf(((FunctionCall) defaultValue).functionName()));
+ default:
+ throw new IllegalStateException("Unknown default value type: " + defaultValue.type());
+ }
+ }
}
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index 51f0d71c8a..18d7775a0c 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -35,8 +35,14 @@ import java.util.concurrent.Flow.Subscription;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Consumer;
import java.util.function.LongFunction;
-import org.apache.ignite.configuration.NamedListView;
-import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
+import org.apache.ignite.internal.catalog.events.AddColumnEventParameters;
+import org.apache.ignite.internal.catalog.events.AlterColumnEventParameters;
+import org.apache.ignite.internal.catalog.events.CatalogEvent;
+import org.apache.ignite.internal.catalog.events.CatalogEventParameters;
+import org.apache.ignite.internal.catalog.events.CreateTableEventParameters;
+import org.apache.ignite.internal.catalog.events.DropColumnEventParameters;
import org.apache.ignite.internal.causality.IncrementalVersionedValue;
import org.apache.ignite.internal.logger.IgniteLogger;
import org.apache.ignite.internal.logger.Loggers;
@@ -78,56 +84,97 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
/** Prevents double stopping of the component. */
private final AtomicBoolean stopGuard = new AtomicBoolean();
- /** Tables configuration. */
- private final TablesConfiguration tablesCfg;
-
/** Versioned store for tables by name. */
private final IncrementalVersionedValue<Map<Integer, SchemaRegistryImpl>> registriesVv;
/** Meta storage manager. */
private final MetaStorageManager metastorageMgr;
+ /** Catalog manager. */
+ private final CatalogManager catalogManager;
+
/** Constructor. */
public SchemaManager(
Consumer<LongFunction<CompletableFuture<?>>> registry,
- TablesConfiguration tablesCfg,
- MetaStorageManager metastorageMgr
- ) {
+ MetaStorageManager metastorageMgr,
+ CatalogManager catalogManager) {
this.registriesVv = new IncrementalVersionedValue<>(registry, HashMap::new);
- this.tablesCfg = tablesCfg;
+ this.catalogManager = catalogManager;
this.metastorageMgr = metastorageMgr;
}
/** {@inheritDoc} */
@Override
public void start() {
- tablesCfg.tables().any().columns().listen(this::onSchemaChange);
+ catalogManager.listen(CatalogEvent.TABLE_CREATE,
+ (params, ex) -> onSchemaChange((CreateTableEventParameters) params).thenApply(ignore -> false));
+ catalogManager.listen(CatalogEvent.TABLE_ALTER,
+ (params, ex) -> onSchemaChange(params).thenApply(ignore -> false));
}
/**
* Listener of schema configuration changes.
*
- * @param ctx Configuration context.
+ * @param evt Event parameters.
* @return A future.
*/
- private CompletableFuture<?> onSchemaChange(ConfigurationNotificationEvent<NamedListView<ColumnView>> ctx) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
- }
+ private CompletableFuture<?> onSchemaChange(CreateTableEventParameters evt) {
+ CatalogTableDescriptor tableDescriptor = evt.tableDescriptor();
+
+ int newSchemaVersion = INITIAL_SCHEMA_VERSION;// evt.catalogVersion();
+ int tblId = tableDescriptor.id();
+ SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
+
+ return onSchemaChange(tblId, newSchema, evt.causalityToken());
+ }
+
+ /**
+ * Listener of schema configuration changes.
+ *
+ * @param evt Event parameters.
+ * @return A future.
+ */
+ private CompletableFuture<?> onSchemaChange(CatalogEventParameters evt) {
try {
- ExtendedTableView tblCfg = ctx.newValue(ExtendedTableView.class);
+ CatalogTableDescriptor tableDescriptor;
+
+ if (evt instanceof AddColumnEventParameters) {
+ AddColumnEventParameters params = (AddColumnEventParameters) evt;
- int newSchemaVersion = tblCfg.schemaId();
+ tableDescriptor = catalogManager.table(params.tableId(), params.catalogVersion());
+ } else if (evt instanceof DropColumnEventParameters) {
+ DropColumnEventParameters params = (DropColumnEventParameters) evt;
- int tblId = tblCfg.id();
+ tableDescriptor = catalogManager.table(params.tableId(), params.catalogVersion());
+ } else if (evt instanceof AlterColumnEventParameters) {
+ AlterColumnEventParameters params = (AlterColumnEventParameters) evt;
- if (searchSchemaByVersion(tblId, newSchemaVersion) != null) {
- return completedFuture(null);
+ tableDescriptor = catalogManager.table(params.tableId(), params.catalogVersion());
+ } else {
+ return completedFuture(new UnsupportedOperationException("Unexpected event."));
}
- SchemaDescriptor newSchema = SchemaUtils.prepareSchemaDescriptor(newSchemaVersion, tblCfg);
+ int newSchemaVersion = registriesVv.latest().get(tableDescriptor.id()).lastSchemaVersion() + 1;;
+ SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
+
+ return onSchemaChange(tableDescriptor.id(), newSchema, evt.causalityToken());
+ } catch (Throwable th) {
+ th.printStackTrace();
+ return failedFuture(th);
+ }
+ }
+
+ /**
+ * Update schema.
+ */
+ private CompletableFuture<?> onSchemaChange(int tblId, SchemaDescriptor newSchema, long causalityToken) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
+ }
+
+ try {
// This is intentionally a blocking call to enforce configuration listener execution order. Unfortunately it is not possible
// to execute this method asynchronously, because the schema descriptor is needed to fire the CREATE event as a synchronous part
// of the configuration listener.
@@ -141,24 +188,13 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
return failedFuture(e);
}
- long causalityToken = ctx.storageRevision();
-
- // Fire event early, because dependent listeners have to register VersionedValues' update futures
- var eventParams = new SchemaEventParameters(causalityToken, tblId, newSchema);
-
- fireEvent(SchemaEvent.CREATE, eventParams)
- .whenComplete((v, e) -> {
+ return registriesVv.update(causalityToken,
+ (registries, e) -> inBusyLock(busyLock, () -> {
if (e != null) {
- LOGGER.warn("Error when processing CREATE event", e);
+ return failedFuture(new IgniteInternalException(IgniteStringFormatter.format(
+ "Cannot create a schema for the table [tblId={}, ver={}]", tblId, newSchema.version()), e)
+ );
}
- });
-
- return registriesVv.update(causalityToken, (registries, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(new IgniteInternalException(IgniteStringFormatter.format(
- "Cannot create a schema for the table [tblId={}, ver={}]", tblId, newSchemaVersion), e)
- );
- }
return saveSchemaDescriptor(tblId, newSchema)
.thenApply(t -> registerSchema(tblId, newSchema, registries));
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index a0b5c9bc24..010e7b1e4d 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -40,6 +40,7 @@ import org.apache.calcite.schema.SchemaPlus;
import org.apache.calcite.tools.Frameworks;
import org.apache.calcite.util.Pair;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.descriptors.CatalogSchemaDescriptor;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.index.IndexManager;
@@ -68,6 +69,7 @@ import org.apache.ignite.internal.sql.engine.prepare.PrepareService;
import org.apache.ignite.internal.sql.engine.prepare.PrepareServiceImpl;
import org.apache.ignite.internal.sql.engine.property.PropertiesHelper;
import org.apache.ignite.internal.sql.engine.property.PropertiesHolder;
+import org.apache.ignite.internal.sql.engine.schema.CatalogSqlSchemaManager;
import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManager;
import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManagerImpl;
import org.apache.ignite.internal.sql.engine.session.Session;
@@ -238,14 +240,6 @@ public class SqlQueryProcessor implements QueryProcessor {
msgSrvc
));
- SqlSchemaManagerImpl sqlSchemaManager = new SqlSchemaManagerImpl(
- tableManager,
- schemaManager,
- registry,
- busyLock
- );
-
- sqlSchemaManager.registerListener(prepareSvc);
this.prepareSvc = prepareSvc;
@@ -261,7 +255,8 @@ public class SqlQueryProcessor implements QueryProcessor {
var dependencyResolver = new ExecutionDependencyResolverImpl(executableTableRegistry);
- sqlSchemaManager.registerListener(executableTableRegistry);
+ CatalogSqlSchemaManager sqlSchemaManager = new CatalogSqlSchemaManager(catalogManager, 1000);
+ this.sqlSchemaManager = sqlSchemaManager;
var executionSrvc = registerService(ExecutionServiceImpl.create(
clusterSrvc.topologyService(),
@@ -272,7 +267,8 @@ public class SqlQueryProcessor implements QueryProcessor {
ArrayRowHandler.INSTANCE,
mailboxRegistry,
exchangeService,
- dependencyResolver
+ dependencyResolver,
+ tableManager
));
clusterSrvc.topologyService().addEventHandler(executionSrvc);
@@ -280,14 +276,6 @@ public class SqlQueryProcessor implements QueryProcessor {
this.executionSrvc = executionSrvc;
- registerTableListener(TableEvent.CREATE, new TableCreatedListener(sqlSchemaManager));
- registerTableListener(TableEvent.ALTER, new TableUpdatedListener(sqlSchemaManager));
- registerTableListener(TableEvent.DROP, new TableDroppedListener(sqlSchemaManager));
-
- registerIndexListener(IndexEvent.CREATE, new IndexCreatedListener(sqlSchemaManager));
- registerIndexListener(IndexEvent.DROP, new IndexDroppedListener(sqlSchemaManager));
-
- this.sqlSchemaManager = sqlSchemaManager;
services.forEach(LifecycleAware::start);
}
@@ -416,7 +404,11 @@ public class SqlQueryProcessor implements QueryProcessor {
AtomicReference<InternalTransaction> tx = new AtomicReference<>();
CompletableFuture<AsyncSqlCursor<List<Object>>> stage = start
- .thenCompose(ignored -> {
+ // TODO: wait for latest catalog.
+ .thenCompose(ignore -> tableManager.tablesAsync())
+ .thenCompose(tables -> CompletableFuture.allOf(tables.stream()
+ .map(t -> tableManager.tableAsync(t.name())).toArray(CompletableFuture[]::new))
+ ).thenCompose(ignored -> {
ParsedResult result = parserService.parse(sql);
validateParsedStatement(context, outerTx, result, params);
@@ -427,7 +419,9 @@ public class SqlQueryProcessor implements QueryProcessor {
tx.set(implicitTxRequired ? txManager.begin(!rwOp, null) : outerTx);
- SchemaPlus schema = sqlSchemaManager.schema(schemaName);
+ int catalogVersion = catalogManager.activeCatalogVersion(tx.get().startTimestamp().longValue());
+
+ SchemaPlus schema = sqlSchemaManager.schema(schemaName, catalogVersion);
if (schema == null) {
return CompletableFuture.failedFuture(new SchemaNotFoundException(schemaName));
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
index 65e878da53..ea12b0a4e0 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutionServiceImpl.java
@@ -235,13 +235,13 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
return queryManager.execute(tx, plan);
}
- private BaseQueryContext createQueryContext(UUID queryId, @Nullable String schema, Object[] params) {
+ private BaseQueryContext createQueryContext(UUID queryId, @Nullable String schema, int version, Object[] params) {
return BaseQueryContext.builder()
.queryId(queryId)
.parameters(params)
.frameworkConfig(
Frameworks.newConfigBuilder(FRAMEWORK_CONFIG)
- .defaultSchema(sqlSchemaManager.schema(schema))
+ .defaultSchema(sqlSchemaManager.schema(schema, version))
.build()
)
.logger(LOG)
@@ -423,7 +423,8 @@ public class ExecutionServiceImpl<RowT> implements ExecutionService, TopologyEve
private DistributedQueryManager getOrCreateQueryManager(QueryStartRequest msg) {
return queryManagerMap.computeIfAbsent(msg.queryId(), key -> {
- BaseQueryContext ctx = createQueryContext(key, msg.schema(), msg.parameters());
+ //TODO: change type to int.
+ BaseQueryContext ctx = createQueryContext(key, msg.schema(), (int) msg.schemaVersion(), msg.parameters());
return new DistributedQueryManager(ctx);
});
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
index ed27b83e91..4f75c4e454 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
@@ -68,10 +68,12 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
}
/** {@inheritDoc} */
+ @Deprecated
@Override
public SchemaPlus schema(@Nullable String schema) {
// Should be removed -schema(name, version) must be used instead
- throw new UnsupportedOperationException();
+ // throw new UnsupportedOperationException();
+ return activeSchema(schema, catalogManager.activeCatalogVersion(Long.MAX_VALUE));
}
/** {@inheritDoc} */
@@ -87,7 +89,12 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
@Override
public IgniteTable tableById(int id) {
// Should be removed - this method is used to obtain native types from a table.
- throw new UnsupportedOperationException();
+ //throw new UnsupportedOperationException();
+ CatalogTableDescriptor table = catalogManager.table(id, Long.MAX_VALUE);
+
+ assert table != null;
+
+ return (IgniteTable) activeSchema(DEFAULT_SCHEMA_NAME, Long.MAX_VALUE).getTable(table.name());
}
/** {@inheritDoc} */
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/BaseQueryContext.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/BaseQueryContext.java
index c6a8cc0743..37cae7bd55 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/BaseQueryContext.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/util/BaseQueryContext.java
@@ -52,7 +52,7 @@ import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.sql.engine.QueryCancel;
import org.apache.ignite.internal.sql.engine.metadata.cost.IgniteCostFactory;
import org.apache.ignite.internal.sql.engine.rex.IgniteRexBuilder;
-import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
+import org.apache.ignite.internal.sql.engine.schema.IgniteCatalogSchema;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.util.ArrayUtils;
@@ -222,7 +222,7 @@ public final class BaseQueryContext extends AbstractQueryContext {
}
public long schemaVersion() {
- return Objects.requireNonNull(schema().unwrap(IgniteSchema.class)).schemaVersion();
+ return Objects.requireNonNull(schema().unwrap(IgniteCatalogSchema.class)).version();
}
/**
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
index 955a4223c2..26a355f767 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/MockedStructuresTest.java
@@ -281,7 +281,7 @@ public class MockedStructuresTest extends IgniteAbstractTest {
dataStorageManager.start();
- schemaManager = new SchemaManager(revisionUpdater, tblsCfg, msm);
+ schemaManager = new SchemaManager(revisionUpdater, msm, catalogManager);
schemaManager.start();
@@ -313,7 +313,7 @@ public class MockedStructuresTest extends IgniteAbstractTest {
tblManager = mockManagers();
- idxManager = new IndexManager(tblsCfg, catalogManager, schemaManager, tblManager);
+ idxManager = new IndexManager(catalogManager, schemaManager, tblManager);
idxManager.start();
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index a3fd1e03d1..961486158b 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -47,6 +47,7 @@ import static org.apache.ignite.internal.utils.RebalanceUtil.extractTableId;
import static org.apache.ignite.internal.utils.RebalanceUtil.pendingPartAssignmentsKey;
import static org.apache.ignite.internal.utils.RebalanceUtil.stablePartAssignmentsKey;
import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
+import static org.apache.ignite.lang.IgniteStringFormatter.format;
import it.unimi.dsi.fastutil.ints.Int2ObjectOpenHashMap;
import java.io.IOException;
@@ -100,7 +101,6 @@ import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescript
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.catalog.events.CatalogEvent;
-import org.apache.ignite.internal.catalog.events.CatalogEventParameters;
import org.apache.ignite.internal.catalog.events.CreateTableEventParameters;
import org.apache.ignite.internal.catalog.events.DropTableEventParameters;
import org.apache.ignite.internal.causality.CompletionListener;
@@ -112,7 +112,6 @@ import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.hlc.HybridTimestamp;
import org.apache.ignite.internal.logger.IgniteLogger;
import org.apache.ignite.internal.logger.Loggers;
-import org.apache.ignite.internal.manager.EventListener;
import org.apache.ignite.internal.manager.IgniteComponent;
import org.apache.ignite.internal.manager.Producer;
import org.apache.ignite.internal.metastorage.Entry;
@@ -149,8 +148,7 @@ import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.schema.configuration.TablesChange;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
-import org.apache.ignite.internal.schema.event.SchemaEvent;
-import org.apache.ignite.internal.schema.event.SchemaEventParameters;
+import org.apache.ignite.internal.schema.configuration.storage.DataStorageView;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.storage.MvPartitionStorage;
import org.apache.ignite.internal.storage.RowId;
@@ -534,18 +532,9 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
catalogManager.listen(CatalogEvent.TABLE_CREATE,
(parameters, exception) -> onTableCreate((CreateTableEventParameters) parameters).thenApply(ignore -> false));
- catalogManager.listen(CatalogEvent.TABLE_CREATE,
+ catalogManager.listen(CatalogEvent.TABLE_DROP,
(parameters, exception) -> onTableDelete((DropTableEventParameters) parameters).thenApply(ignore -> false));
- schemaManager.listen(SchemaEvent.CREATE, new EventListener<>() {
- @Override
- public CompletableFuture<Boolean> notify(@NotNull SchemaEventParameters parameters, @Nullable Throwable exception) {
- var eventParameters = new TableEventParameters(parameters.causalityToken(), parameters.tableId());
-
- return fireEvent(TableEvent.ALTER, eventParameters).thenApply(v -> false);
- }
- });
-
addMessageHandler(clusterService.messagingService());
}
@@ -589,13 +578,13 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
/**
* Listener of table create configuration change.
*
- * @param ctx Table configuration context.
+ * @param evt Event parameters.
* @return A future.
*/
- private CompletableFuture<?> onTableCreate(CreateTableEventParameters ctx) {
+ private CompletableFuture<?> onTableCreate(CreateTableEventParameters evt) {
if (!busyLock.enterBusy()) {
fireEvent(TableEvent.CREATE,
- new TableEventParameters(ctx.causalityToken(), ctx.tableDescriptor().id()),
+ new TableEventParameters(evt.causalityToken(), evt.tableDescriptor().id()),
new NodeStoppingException()
);
@@ -603,10 +592,8 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- CatalogTableDescriptor tableDescriptor = ctx.tableDescriptor();
- CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), ctx.catalogVersion());
-
- assert zoneDescriptor != null;
+ CatalogTableDescriptor tableDescriptor = evt.tableDescriptor();
+ CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), evt.catalogVersion());
List<Set<Assignment>> assignments;
@@ -628,7 +615,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
assert !assignments.isEmpty() : "Couldn't create the table with empty assignments.";
CompletableFuture<?> createTableFut = createTableLocally(
- ctx.causalityToken(),
+ evt.causalityToken(),
tableDescriptor,
zoneDescriptor,
assignments
@@ -645,6 +632,10 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
writeTableAssignmentsToMetastore(tableId, assignments);
return createTableFut;
+ } catch (Throwable th) {
+ LOG.warn("Failed to process create table event.", th);
+
+ return failedFuture(th);
} finally {
busyLock.leaveBusy();
}
@@ -676,14 +667,14 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
/**
* Listener of table drop configuration change.
*
- * @param ctx Table configuration context.
+ * @param evt Event parameters.
* @return A future.
*/
- private CompletableFuture<?> onTableDelete(DropTableEventParameters ctx) {
+ private CompletableFuture<?> onTableDelete(DropTableEventParameters evt) {
if (!busyLock.enterBusy()) {
fireEvent(
TableEvent.DROP,
- new TableEventParameters(ctx.causalityToken(), ctx.tableId()),
+ new TableEventParameters(evt.causalityToken(), evt.tableId()),
new NodeStoppingException()
);
@@ -691,10 +682,16 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- CatalogTableDescriptor tableDescriptor = catalogManager.table(ctx.tableId(), ctx.catalogVersion() - 1);
- CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), ctx.catalogVersion() - 1);
+ int previousCatalogVersion = evt.catalogVersion() - 1;
+
+ CatalogTableDescriptor tableDescriptor = catalogManager.table(evt.tableId(), previousCatalogVersion);
+ CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), previousCatalogVersion);
- dropTableLocally(ctx.causalityToken(), tableDescriptor, zoneDescriptor);
+ dropTableLocally(evt.causalityToken(), tableDescriptor, zoneDescriptor);
+ } catch (Throwable th) {
+ LOG.warn("Failed to process drop table event.", th);
+
+ return failedFuture(th);
} finally {
busyLock.leaveBusy();
}
@@ -723,7 +720,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
// Empty assignments might be a valid case if tables are created from within cluster init HOCON
// configuration, which is not supported now.
- assert newAssignments != null : IgniteStringFormatter.format("Table [id={}] has empty assignments.", tableId);
+ assert newAssignments != null : format("Table [id={}] has empty assignments.", tableId);
int partitions = newAssignments.size();
@@ -1284,18 +1281,22 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
pendingTables.put(tableId, table);
startedTables.put(tableId, table);
- tablesById(causalityToken).thenAccept(ignored -> inBusyLock(busyLock, () -> {
+ createPartsFut.thenAccept(ignored -> inBusyLock(busyLock, () -> {
pendingTables.remove(tableId);
}));
- tablesById(causalityToken)
- .thenRun(() -> inBusyLock(busyLock, () -> completeApiCreateFuture(table)));
+ createPartsFut
+ .thenRunAsync(() -> inBusyLock(busyLock, () -> completeApiCreateFuture(table)));
// TODO should be reworked in IGNITE-16763
// We use the event notification future as the result so that dependent components can complete the schema updates.
// TODO: https://issues.apache.org/jira/browse/IGNITE-19913 Possible performance degradation.
- return allOf(createPartsFut, fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId)));
+ CompletableFuture<?> eventFut = fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
+
+ // TODO: investigate why createParts and eventFutures hangs.
+// return allOf(createPartsFut, eventFut);
+ return completedFuture(false);
}
/**
@@ -1504,6 +1505,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
.changeColocationColumns(colocationKeys0.toArray(String[]::new)));
};
+
return catalogManager.createTable(parameters)
.thenApply(ignore -> catalogManager.table(tableName, Long.MAX_VALUE).id())
.thenCompose(tableId -> createTableAsyncInternal(tableId, tableName, zoneName, tblChanger));
@@ -2849,8 +2851,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
if (primaryCols.contains(colName)) {
- throw new SqlException(STMT_VALIDATION_ERR, IgniteStringFormatter
- .format("Can`t delete column, belongs to primary key: [name={}]", colName));
+ throw new SqlException(STMT_VALIDATION_ERR, format("Can`t delete column, belongs to primary key: [name={}]", colName));
}
}
diff --git a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
index 22aa333a72..7fc676d9b3 100644
--- a/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
+++ b/modules/table/src/test/java/org/apache/ignite/internal/table/distributed/TableManagerTest.java
@@ -23,7 +23,6 @@ import static org.apache.ignite.internal.distributionzones.DistributionZoneManag
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.getZoneById;
import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
-import static org.apache.ignite.internal.testframework.IgniteTestUtils.cause;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willSucceedFast;
import static org.hamcrest.MatcherAssert.assertThat;
@@ -38,8 +37,6 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
-import static org.mockito.ArgumentMatchers.anyLong;
-import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
@@ -71,10 +68,7 @@ import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.catalog.CatalogServiceImpl;
import org.apache.ignite.internal.catalog.ClockWaiter;
import org.apache.ignite.internal.catalog.commands.AlterTableAddColumnParams;
-import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
-import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
import org.apache.ignite.internal.catalog.commands.DropTableParams;
-import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.catalog.storage.UpdateLogImpl;
@@ -85,7 +79,6 @@ import org.apache.ignite.internal.configuration.notifications.ConfigurationStora
import org.apache.ignite.internal.configuration.testframework.ConfigurationExtension;
import org.apache.ignite.internal.configuration.testframework.InjectConfiguration;
import org.apache.ignite.internal.configuration.testframework.InjectRevisionListenerHolder;
-import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.hlc.HybridClockImpl;
@@ -867,8 +860,6 @@ public class TableManagerTest extends IgniteAbstractTest {
when(vaultManager.get(any(ByteArray.class))).thenReturn(completedFuture(null));
when(vaultManager.put(any(ByteArray.class), any(byte[].class))).thenReturn(completedFuture(null));
- //TODO IGNITE-19082 drop mocked catalog manager.
-
CatalogManager catalogManager = new CatalogServiceImpl(
new UpdateLogImpl(StandaloneMetaStorageManager.create(vaultManager)),
new ClockWaiter("note", new HybridClockImpl()));
@@ -893,7 +884,7 @@ public class TableManagerTest extends IgniteAbstractTest {
workDir,
msm,
catalogManager,
- sm = new SchemaManager(revisionUpdater, tblsCfg, msm),
+ sm = new SchemaManager(revisionUpdater, msm, catalogManager),
budgetView -> new LocalLogStorageFactory(),
new HybridClockImpl(),
new OutgoingSnapshotsManager(clusterService.messagingService()),
[ignite-3] 10/19: wip. switch to CalciteSqlSchemaManager. Drop dead code.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 1d4bb280927dbd011603fc62f41b4ad77c739082
Author: amashenkov <an...@gmail.com>
AuthorDate: Wed Jul 19 22:41:52 2023 +0300
wip. switch to CalciteSqlSchemaManager.
Drop dead code.
---
.../ignite/internal/schema/SchemaManager.java | 17 +-
.../internal/sql/engine/SqlQueryProcessor.java | 123 -----
.../engine/exec/ExecutableTableRegistryImpl.java | 1 -
.../sql/engine/exec/LogicalRelImplementor.java | 27 +-
.../sql/engine/exec/rel/IndexScanNode.java | 18 +-
.../internal/sql/engine/rel/AbstractIndexScan.java | 8 +-
.../internal/sql/engine/rel/IgniteIndexScan.java | 6 +-
.../engine/rel/logical/IgniteLogicalIndexScan.java | 5 +-
.../sql/engine/rule/LogicalScanConverterRule.java | 6 +-
.../sql/engine/rule/logical/ExposeIndexRule.java | 10 +-
.../engine/rule/logical/LogicalOrToUnionRule.java | 4 +-
.../sql/engine/schema/CatalogSqlSchemaManager.java | 20 +-
.../internal/sql/engine/schema/IgniteIndex.java | 167 ++----
.../sql/engine/schema/IgniteSchemaIndex.java | 143 -----
.../sql/engine/schema/IgniteSchemaTable.java | 32 +-
.../sql/engine/schema/IgniteTableImpl.java | 307 -----------
.../sql/engine/schema/SqlSchemaManagerImpl.java | 588 ---------------------
.../internal/sql/engine/trait/TraitUtils.java | 105 ++--
.../exec/ExecutionDependencyResolverSelfTest.java | 7 +-
.../exec/rel/IndexScanNodeExecutionTest.java | 60 ++-
.../engine/exec/schema/SqlSchemaManagerTest.java | 328 ------------
.../sql/engine/framework/TestBuilders.java | 89 +++-
.../internal/sql/engine/framework/TestIndex.java | 34 +-
.../planner/AbstractAggregatePlannerTest.java | 3 +-
.../sql/engine/planner/AbstractPlannerTest.java | 3 +-
.../sql/engine/planner/HashIndexPlannerTest.java | 8 +-
.../sql/engine/planner/ImplicitCastsTest.java | 5 +-
.../engine/planner/JoinColocationPlannerTest.java | 16 +-
.../internal/sql/engine/planner/PlannerTest.java | 6 +-
.../planner/ProjectFilterScanMergePlannerTest.java | 9 +-
.../engine/schema/CatalogSqlSchemaManagerTest.java | 9 +-
31 files changed, 346 insertions(+), 1818 deletions(-)
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index 85edc90e35..41f13823e2 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -107,9 +107,9 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
@Override
public void start() {
catalogManager.listen(CatalogEvent.TABLE_CREATE,
- (params, ex) -> onSchemaChange((CreateTableEventParameters) params).thenApply(ignore -> false));
+ (params, ex) -> onTableCreated((CreateTableEventParameters) params).thenApply(ignore -> false));
catalogManager.listen(CatalogEvent.TABLE_ALTER,
- (params, ex) -> onSchemaChange(params).thenApply(ignore -> false));
+ (params, ex) -> onTableChanged(params).thenApply(ignore -> false));
}
/**
@@ -118,7 +118,7 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
* @param evt Event parameters.
* @return A future.
*/
- private CompletableFuture<?> onSchemaChange(CreateTableEventParameters evt) {
+ private CompletableFuture<?> onTableCreated(CreateTableEventParameters evt) {
CatalogTableDescriptor tableDescriptor = evt.tableDescriptor();
int newSchemaVersion = INITIAL_SCHEMA_VERSION; // evt.catalogVersion();
@@ -135,7 +135,11 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
* @param evt Event parameters.
* @return A future.
*/
- private CompletableFuture<?> onSchemaChange(CatalogEventParameters evt) {
+ private CompletableFuture<?> onTableChanged(CatalogEventParameters evt) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
+ }
+
try {
CatalogTableDescriptor tableDescriptor;
@@ -155,14 +159,17 @@ public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters>
return completedFuture(new UnsupportedOperationException("Unexpected event."));
}
- int newSchemaVersion = registriesVv.latest().get(tableDescriptor.id()).lastSchemaVersion() + 1;;
+ int newSchemaVersion = registriesVv.latest().get(tableDescriptor.id()).lastSchemaVersion() + 1;
SchemaDescriptor newSchema = CatalogDescriptorUtils.convert(newSchemaVersion, tableDescriptor);
return onSchemaChange(tableDescriptor.id(), newSchema, evt.causalityToken());
} catch (Throwable th) {
+ //TODO: drop debug print.
th.printStackTrace();
return failedFuture(th);
+ } finally {
+ busyLock.leaveBusy();
}
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index 6d5cd4c58a..4f64f709d3 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -44,7 +44,6 @@ import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.hlc.HybridClock;
import org.apache.ignite.internal.index.IndexManager;
import org.apache.ignite.internal.index.event.IndexEvent;
-import org.apache.ignite.internal.index.event.IndexEventParameters;
import org.apache.ignite.internal.logger.IgniteLogger;
import org.apache.ignite.internal.logger.Loggers;
import org.apache.ignite.internal.manager.Event;
@@ -70,7 +69,6 @@ import org.apache.ignite.internal.sql.engine.property.PropertiesHelper;
import org.apache.ignite.internal.sql.engine.property.PropertiesHolder;
import org.apache.ignite.internal.sql.engine.schema.CatalogSqlSchemaManager;
import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManager;
-import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManagerImpl;
import org.apache.ignite.internal.sql.engine.session.Session;
import org.apache.ignite.internal.sql.engine.session.SessionId;
import org.apache.ignite.internal.sql.engine.session.SessionInfo;
@@ -86,7 +84,6 @@ import org.apache.ignite.internal.sql.engine.util.TypeUtils;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.table.distributed.TableManager;
import org.apache.ignite.internal.table.event.TableEvent;
-import org.apache.ignite.internal.table.event.TableEventParameters;
import org.apache.ignite.internal.tx.InternalTransaction;
import org.apache.ignite.internal.tx.TxManager;
import org.apache.ignite.internal.util.IgniteSpinBusyLock;
@@ -96,7 +93,6 @@ import org.apache.ignite.lang.NodeStoppingException;
import org.apache.ignite.lang.SchemaNotFoundException;
import org.apache.ignite.network.ClusterService;
import org.apache.ignite.sql.SqlException;
-import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
/**
@@ -355,18 +351,6 @@ public class SqlQueryProcessor implements QueryProcessor {
return service;
}
- private void registerTableListener(TableEvent evt, AbstractTableEventListener lsnr) {
- evtLsnrs.add(Pair.of(evt, lsnr));
-
- tableManager.listen(evt, lsnr);
- }
-
- private void registerIndexListener(IndexEvent evt, AbstractIndexEventListener lsnr) {
- evtLsnrs.add(Pair.of(evt, lsnr));
-
- indexManager.listen(evt, lsnr);
- }
-
private CompletableFuture<AsyncSqlCursor<List<Object>>> querySingle0(
SessionId sessionId,
QueryContext context,
@@ -485,113 +469,6 @@ public class SqlQueryProcessor implements QueryProcessor {
return stage;
}
- private abstract static class AbstractTableEventListener implements EventListener<TableEventParameters> {
- protected final SqlSchemaManagerImpl schemaHolder;
-
- private AbstractTableEventListener(SqlSchemaManagerImpl schemaHolder) {
- this.schemaHolder = schemaHolder;
- }
- }
-
- private abstract static class AbstractIndexEventListener implements EventListener<IndexEventParameters> {
- protected final SqlSchemaManagerImpl schemaHolder;
-
- private AbstractIndexEventListener(SqlSchemaManagerImpl schemaHolder) {
- this.schemaHolder = schemaHolder;
- }
- }
-
- private static class TableCreatedListener extends AbstractTableEventListener {
- private TableCreatedListener(SqlSchemaManagerImpl schemaHolder) {
- super(schemaHolder);
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<Boolean> notify(@NotNull TableEventParameters parameters, @Nullable Throwable exception) {
- return schemaHolder.onTableCreated(
- // TODO: https://issues.apache.org/jira/browse/IGNITE-17694 Hardcoded schemas
- DEFAULT_SCHEMA_NAME,
- parameters.tableId(),
- parameters.causalityToken()
- )
- .thenApply(v -> false);
- }
- }
-
- private static class TableUpdatedListener extends AbstractTableEventListener {
- private TableUpdatedListener(SqlSchemaManagerImpl schemaHolder) {
- super(schemaHolder);
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<Boolean> notify(@NotNull TableEventParameters parameters, @Nullable Throwable exception) {
- return schemaHolder.onTableUpdated(
- // TODO: https://issues.apache.org/jira/browse/IGNITE-17694 Hardcoded schemas
- DEFAULT_SCHEMA_NAME,
- parameters.tableId(),
- parameters.causalityToken()
- )
- .thenApply(v -> false);
- }
- }
-
- private static class TableDroppedListener extends AbstractTableEventListener {
- private TableDroppedListener(SqlSchemaManagerImpl schemaHolder) {
- super(schemaHolder);
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<Boolean> notify(@NotNull TableEventParameters parameters, @Nullable Throwable exception) {
- return schemaHolder.onTableDropped(
- // TODO: https://issues.apache.org/jira/browse/IGNITE-17694 Hardcoded schemas
- DEFAULT_SCHEMA_NAME,
- parameters.tableId(),
- parameters.causalityToken()
- )
- .thenApply(v -> false);
- }
- }
-
- private static class IndexDroppedListener extends AbstractIndexEventListener {
- private IndexDroppedListener(SqlSchemaManagerImpl schemaHolder) {
- super(schemaHolder);
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<Boolean> notify(@NotNull IndexEventParameters parameters, @Nullable Throwable exception) {
- return schemaHolder.onIndexDropped(
- // TODO: https://issues.apache.org/jira/browse/IGNITE-17694 Hardcoded schemas
- DEFAULT_SCHEMA_NAME,
- parameters.tableId(),
- parameters.indexId(),
- parameters.causalityToken()
- )
- .thenApply(v -> false);
- }
- }
-
- private static class IndexCreatedListener extends AbstractIndexEventListener {
- private IndexCreatedListener(SqlSchemaManagerImpl schemaHolder) {
- super(schemaHolder);
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<Boolean> notify(@NotNull IndexEventParameters parameters, @Nullable Throwable exception) {
- return schemaHolder.onIndexCreated(
- parameters.tableId(),
- parameters.indexId(),
- parameters.indexDescriptor(),
- parameters.causalityToken()
- )
- .thenApply(v -> false);
- }
- }
-
/** Returns {@code true} if this is data modification operation. */
private static boolean dataModificationOp(ParsedResult parsedResult) {
return parsedResult.queryType() == SqlQueryType.DML;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
index 096a0989d9..92b254855d 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ExecutableTableRegistryImpl.java
@@ -18,7 +18,6 @@
package org.apache.ignite.internal.sql.engine.exec;
import com.github.benmanes.caffeine.cache.Caffeine;
-import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentMap;
import org.apache.ignite.internal.hlc.HybridClock;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
index 464a0c0d5a..d1bfb1187f 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/LogicalRelImplementor.java
@@ -32,12 +32,14 @@ import java.util.function.Predicate;
import java.util.function.Supplier;
import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.rel.RelCollation;
+import org.apache.calcite.rel.RelFieldCollation;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.rel.core.Intersect;
import org.apache.calcite.rel.core.JoinRelType;
import org.apache.calcite.rel.core.Minus;
import org.apache.calcite.rel.core.Spool;
import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.rex.RexInputRef;
import org.apache.calcite.rex.RexLiteral;
import org.apache.calcite.rex.RexNode;
@@ -343,11 +345,15 @@ public class LogicalRelImplementor<RowT> implements IgniteRelVisitor<Node<RowT>>
if (searchBounds != null) {
Comparator<RowT> searchRowComparator = null;
- if (idx.collations() != null) {
- searchRowComparator = expressionFactory.comparator(TraitUtils.createCollation(idx.collations()));
+ // TODO: cache comparator?
+ if (idx.type() != Type.HASH) {
+ searchRowComparator = expressionFactory.comparator(idx.collation());
}
- ranges = expressionFactory.ranges(searchBounds, idx.getRowType(typeFactory, tbl.descriptor()), searchRowComparator);
+ // TODO: cache rowtype?
+ RelDataType idxRowType = getIndexRowType(typeFactory, tbl, idx);
+
+ ranges = expressionFactory.ranges(searchBounds, idxRowType, searchRowComparator);
}
RelCollation outputCollation = rel.collation();
@@ -361,13 +367,14 @@ public class LogicalRelImplementor<RowT> implements IgniteRelVisitor<Node<RowT>>
}
ColocationGroup group = ctx.group(rel.sourceId());
- Comparator<RowT> comp = idx.type() == Type.SORTED ? ctx.expressionFactory().comparator(outputCollation) : null;
+ Comparator<RowT> comp = idx.type() == IgniteIndex.Type.SORTED ? ctx.expressionFactory().comparator(outputCollation) : null;
if (!group.nodeNames().contains(ctx.localNode().name())) {
return new ScanNode<>(ctx, Collections.emptyList());
}
return new IndexScanNode<>(
+ idx.id(),
ctx,
ctx.rowHandler().factory(ctx.getTypeFactory(), rowType),
idx,
@@ -382,6 +389,18 @@ public class LogicalRelImplementor<RowT> implements IgniteRelVisitor<Node<RowT>>
);
}
+ private static RelDataType getIndexRowType(IgniteTypeFactory typeFactory, IgniteTable tbl, IgniteIndex idx) {
+ RelDataTypeFactory.Builder b = new RelDataTypeFactory.Builder(typeFactory);
+
+ RelDataType rt = tbl.getRowType(typeFactory);
+ for (RelFieldCollation fc : idx.collation().getFieldCollations()) {
+ b.add(rt.getFieldList().get(fc.getFieldIndex()));
+ }
+
+ RelDataType idxRowType = b.build();
+ return idxRowType;
+ }
+
/** {@inheritDoc} */
@Override
public Node<RowT> visit(IgniteTableScan rel) {
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNode.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNode.java
index 55ad834cdf..ecc3bb04bc 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNode.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNode.java
@@ -25,12 +25,15 @@ import java.util.List;
import java.util.concurrent.Flow.Publisher;
import java.util.function.Function;
import java.util.function.Predicate;
+import java.util.stream.Collectors;
+import org.apache.calcite.rel.RelFieldCollation;
import org.apache.ignite.internal.sql.engine.exec.ExecutionContext;
import org.apache.ignite.internal.sql.engine.exec.RowHandler;
import org.apache.ignite.internal.sql.engine.exec.ScannableTable;
import org.apache.ignite.internal.sql.engine.exec.exp.RangeCondition;
import org.apache.ignite.internal.sql.engine.exec.exp.RangeIterable;
import org.apache.ignite.internal.sql.engine.metadata.PartitionWithTerm;
+import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.sql.engine.util.Commons;
@@ -42,6 +45,8 @@ import org.jetbrains.annotations.Nullable;
* Execution node for index scan. Provide result of index scan by given index, partitions and range conditions.
*/
public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
+ private final int indexId;
+
/** Schema index. */
private final IgniteIndex schemaIndex;
@@ -59,6 +64,8 @@ public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
private final @Nullable Comparator<RowT> comp;
+ private final TableDescriptor tableDescriptor;
+
/**
* Constructor.
*
@@ -73,6 +80,7 @@ public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
* @param requiredColumns Optional set of column of interest.
*/
public IndexScanNode(
+ int indexId,
ExecutionContext<RowT> ctx,
RowHandler.RowFactory<RowT> rowFactory,
IgniteIndex schemaIndex,
@@ -89,6 +97,7 @@ public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
assert partsWithTerms != null && !partsWithTerms.isEmpty();
+ this.indexId = indexId;
this.schemaIndex = schemaIndex;
this.table = table;
this.partsWithTerms = partsWithTerms;
@@ -96,6 +105,7 @@ public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
this.rangeConditions = rangeConditions;
this.comp = comp;
this.factory = rowFactory;
+ this.tableDescriptor = tableDescriptor;
}
/** {@inheritDoc} */
@@ -123,8 +133,12 @@ public class IndexScanNode<RowT> extends StorageScanNode<RowT> {
}
private Publisher<RowT> partitionPublisher(PartitionWithTerm partWithTerm, @Nullable RangeCondition<RowT> cond) {
- int indexId = schemaIndex.id();
- List<String> columns = schemaIndex.columns();
+ //TODO: optimize this
+ List<String> columns = schemaIndex.collation().getFieldCollations().stream()
+ .map(RelFieldCollation::getFieldIndex)
+ .map(tableDescriptor::columnDescriptor)
+ .map(ColumnDescriptor::name)
+ .collect(Collectors.toList());
ExecutionContext<RowT> ctx = context();
switch (schemaIndex.type()) {
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/AbstractIndexScan.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/AbstractIndexScan.java
index d7e416f6ce..22fb6a21d2 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/AbstractIndexScan.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/AbstractIndexScan.java
@@ -53,7 +53,7 @@ public abstract class AbstractIndexScan extends ProjectableFilterableTableScan {
protected final @Nullable List<SearchBounds> searchBounds;
- protected final IgniteIndex.Type type;
+ protected final Type type;
/**
* Constructor used for deserialization.
@@ -63,7 +63,7 @@ public abstract class AbstractIndexScan extends ProjectableFilterableTableScan {
AbstractIndexScan(RelInput input) {
super(input);
idxName = input.getString("index");
- type = input.getEnum("type", IgniteIndex.Type.class);
+ type = input.getEnum("type", Type.class);
searchBounds = ((RelInputEx) input).getSearchBounds("searchBounds");
}
@@ -77,7 +77,7 @@ public abstract class AbstractIndexScan extends ProjectableFilterableTableScan {
List<RelHint> hints,
RelOptTable table,
String idxName,
- IgniteIndex.Type type,
+ Type type,
@Nullable List<RexNode> proj,
@Nullable RexNode cond,
@Nullable List<SearchBounds> searchBounds,
@@ -126,7 +126,7 @@ public abstract class AbstractIndexScan extends ProjectableFilterableTableScan {
double cost = 0;
- if (type == Type.HASH) {
+ if (type == IgniteIndex.Type.HASH) {
boolean notExact = (searchBounds() == null)
|| searchBounds().stream().anyMatch(bound -> bound.type() == SearchBounds.Type.RANGE);
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/IgniteIndexScan.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/IgniteIndexScan.java
index 52d2ff9d3a..ad29121527 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/IgniteIndexScan.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/IgniteIndexScan.java
@@ -29,7 +29,7 @@ import org.apache.calcite.rel.RelWriter;
import org.apache.calcite.rex.RexNode;
import org.apache.calcite.util.ImmutableBitSet;
import org.apache.ignite.internal.sql.engine.prepare.bounds.SearchBounds;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.jetbrains.annotations.Nullable;
/**
@@ -78,7 +78,7 @@ public class IgniteIndexScan extends AbstractIndexScan implements SourceAwareIgn
RelTraitSet traits,
RelOptTable tbl,
String idxName,
- IgniteIndex.Type type,
+ Type type,
RelCollation collation,
@Nullable List<RexNode> proj,
@Nullable RexNode cond,
@@ -109,7 +109,7 @@ public class IgniteIndexScan extends AbstractIndexScan implements SourceAwareIgn
RelTraitSet traits,
RelOptTable tbl,
String idxName,
- IgniteIndex.Type type,
+ Type type,
RelCollation collation,
@Nullable List<RexNode> proj,
@Nullable RexNode cond,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/logical/IgniteLogicalIndexScan.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/logical/IgniteLogicalIndexScan.java
index d76a263c68..631ee4d053 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/logical/IgniteLogicalIndexScan.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rel/logical/IgniteLogicalIndexScan.java
@@ -30,7 +30,6 @@ import org.apache.ignite.internal.sql.engine.rel.AbstractIndexScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
-import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.Commons;
import org.apache.ignite.internal.sql.engine.util.RexUtils;
@@ -54,7 +53,7 @@ public class IgniteLogicalIndexScan extends AbstractIndexScan {
IgniteTable tbl = table.unwrap(IgniteTable.class);
IgniteTypeFactory typeFactory = Commons.typeFactory(cluster);
IgniteIndex index = tbl.getIndex(idxName);
- RelCollation collation = TraitUtils.createCollation(index.columns(), index.collations(), tbl.descriptor());
+ RelCollation collation = index.collation();
List<SearchBounds> searchBounds;
if (index.type() == Type.HASH) {
@@ -111,7 +110,7 @@ public class IgniteLogicalIndexScan extends AbstractIndexScan {
RelTraitSet traits,
RelOptTable tbl,
String idxName,
- IgniteIndex.Type type,
+ Type type,
@Nullable List<RexNode> proj,
@Nullable RexNode cond,
@Nullable List<SearchBounds> searchBounds,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/LogicalScanConverterRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/LogicalScanConverterRule.java
index 836cfe19de..359b2b15fe 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/LogicalScanConverterRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/LogicalScanConverterRule.java
@@ -40,9 +40,7 @@ import org.apache.ignite.internal.sql.engine.rel.ProjectableFilterableTableScan;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
-import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
/**
* LogicalScanConverterRule.
@@ -65,8 +63,8 @@ public abstract class LogicalScanConverterRule<T extends ProjectableFilterableTa
IgniteIndex index = table.getIndex(rel.indexName());
RelDistribution distribution = table.distribution();
- RelCollation collation = TraitUtils.createCollation(index.columns(), index.collations(), table.descriptor());
- RelCollation outputCollation = index.type() == Type.HASH ? RelCollations.EMPTY : collation;
+ RelCollation collation = index.collation();
+ RelCollation outputCollation = index.type() == IgniteIndex.Type.HASH ? RelCollations.EMPTY : collation;
if (rel.projects() != null || rel.requiredColumns() != null) {
Mappings.TargetMapping mapping = createMapping(
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
index d9078b4d4c..f31c9e6b07 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/ExposeIndexRule.java
@@ -33,8 +33,6 @@ import org.apache.ignite.internal.sql.engine.prepare.bounds.SearchBounds;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
-import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaTable;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.immutables.value.Value;
@@ -52,7 +50,7 @@ public class ExposeIndexRule extends RelRule<ExposeIndexRule.Config> {
private static boolean preMatch(IgniteLogicalTableScan scan) {
// has indexes to expose
- return !scan.getTable().unwrap(IgniteSchemaTable.class).getIndexes().isEmpty();
+ return !scan.getTable().unwrap(IgniteTable.class).indexes().isEmpty();
}
/** {@inheritDoc} */
@@ -62,12 +60,12 @@ public class ExposeIndexRule extends RelRule<ExposeIndexRule.Config> {
RelOptCluster cluster = scan.getCluster();
RelOptTable optTable = scan.getTable();
- IgniteSchemaTable igniteTable = optTable.unwrap(IgniteSchemaTable.class);
+ IgniteTable igniteTable = optTable.unwrap(IgniteTable.class);
List<RexNode> proj = scan.projects();
RexNode condition = scan.condition();
ImmutableBitSet requiredCols = scan.requiredColumns();
- List<IgniteLogicalIndexScan> indexes = igniteTable.getIndexes().values().stream()
+ List<IgniteLogicalIndexScan> indexes = igniteTable.indexes().values().stream()
.map(idx -> idx.toRel(cluster, optTable, proj, condition, requiredCols))
.filter(idx -> filter(igniteTable, idx.indexName(), idx.searchBounds()))
.collect(Collectors.toList());
@@ -88,7 +86,7 @@ public class ExposeIndexRule extends RelRule<ExposeIndexRule.Config> {
private static boolean filter(IgniteTable table, String idxName, List<SearchBounds> searchBounds) {
IgniteIndex index = table.getIndex(idxName);
- return index.type() == Type.SORTED || (searchBounds != null
+ return index.type() == IgniteIndex.Type.SORTED || (searchBounds != null
&& searchBounds.stream().noneMatch(bound -> bound.type() == SearchBounds.Type.RANGE));
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
index eb2bb7d442..0612c7612e 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/rule/logical/LogicalOrToUnionRule.java
@@ -37,7 +37,7 @@ import org.apache.calcite.sql.SqlKind;
import org.apache.calcite.tools.RelBuilder;
import org.apache.calcite.util.mapping.Mappings;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchemaTable;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.Commons;
@@ -137,7 +137,7 @@ public class LogicalOrToUnionRule extends RelRule<LogicalOrToUnionRule.Config> {
BitSet idxsFirstFields = new BitSet(fieldCnt);
- for (IgniteSchemaIndex idx : tbl.getIndexes().values()) {
+ for (IgniteIndex idx : tbl.indexes().values()) {
List<RelFieldCollation> fieldCollations = idx.collation().getFieldCollations();
if (!CollectionUtils.nullOrEmpty(fieldCollations)) {
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
index 4f75c4e454..8dbd7e8fad 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManager.java
@@ -50,6 +50,7 @@ import org.apache.ignite.internal.schema.DefaultValueGenerator;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
+import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
import org.jetbrains.annotations.Nullable;
/**
@@ -72,8 +73,7 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
@Override
public SchemaPlus schema(@Nullable String schema) {
// Should be removed -schema(name, version) must be used instead
- // throw new UnsupportedOperationException();
- return activeSchema(schema, catalogManager.activeCatalogVersion(Long.MAX_VALUE));
+ throw new UnsupportedOperationException();
}
/** {@inheritDoc} */
@@ -129,7 +129,7 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
tableDescriptorMap.put(tableDescriptor.id(), descriptorImpl);
}
- Map<Integer, Map<String, IgniteSchemaIndex>> schemaTableIndexes = new HashMap<>(descriptor.indexes().length);
+ Map<Integer, Map<String, IgniteIndex>> schemaTableIndexes = new HashMap<>(descriptor.indexes().length);
// Assemble indexes as they are required by tables.
for (CatalogIndexDescriptor indexDescriptor : descriptor.indexes()) {
@@ -138,19 +138,20 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
assert tableDescriptorImpl != null : "Table is not found in schema: " + tableId;
String indexName = indexDescriptor.name();
- Map<String, IgniteSchemaIndex> tableIndexes = schemaTableIndexes.computeIfAbsent(tableId, id -> new LinkedHashMap<>());
+ Map<String, IgniteIndex> tableIndexes = schemaTableIndexes.computeIfAbsent(tableId, id -> new LinkedHashMap<>());
Type type;
if (indexDescriptor instanceof CatalogSortedIndexDescriptor) {
- type = Type.SORTED;
+ type = IgniteIndex.Type.SORTED;
} else if (indexDescriptor instanceof CatalogHashIndexDescriptor) {
- type = Type.HASH;
+ type = IgniteIndex.Type.HASH;
} else {
throw new IllegalArgumentException("Unexpected index type: " + indexDescriptor);
}
- RelCollation indexCollation = IgniteSchemaIndex.createIndexCollation(indexDescriptor, tableDescriptorImpl);
- IgniteSchemaIndex schemaIndex = new IgniteSchemaIndex(indexName, type, tableDescriptorImpl.distribution(), indexCollation);
+ RelCollation indexCollation = TraitUtils.createIndexCollation(indexDescriptor, tableDescriptorImpl);
+ IgniteDistribution distribution = tableDescriptorImpl.distribution();
+ IgniteIndex schemaIndex = new IgniteIndex(indexDescriptor.id(), indexName, type, distribution, indexCollation);
tableIndexes.put(indexName, schemaIndex);
schemaTableIndexes.put(tableId, tableIndexes);
@@ -164,7 +165,7 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
assert descriptorImpl != null;
IgniteStatistic statistic = new IgniteStatistic(() -> 0.0d, descriptorImpl.distribution());
- Map<String, IgniteSchemaIndex> tableIndexMap = schemaTableIndexes.getOrDefault(tableId, Collections.emptyMap());
+ Map<String, IgniteIndex> tableIndexMap = schemaTableIndexes.getOrDefault(tableId, Collections.emptyMap());
IgniteSchemaTable schemaTable = new IgniteSchemaTable(tableName, tableId, version, descriptorImpl, statistic, tableIndexMap);
@@ -218,7 +219,6 @@ public class CatalogSqlSchemaManager implements SqlSchemaManager {
defaultValueSupplier = null;
}
-
CatalogColumnDescriptor columnDescriptor = new CatalogColumnDescriptor(
col.name(),
key,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteIndex.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteIndex.java
index 244ccc3784..34557790d3 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteIndex.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteIndex.java
@@ -17,23 +17,20 @@
package org.apache.ignite.internal.sql.engine.schema;
-import static org.apache.ignite.internal.sql.engine.util.TypeUtils.native2relationalType;
-
-import java.util.ArrayList;
import java.util.List;
-import java.util.Objects;
-import org.apache.calcite.rel.type.RelDataType;
-import org.apache.calcite.rel.type.RelDataTypeFactory;
-import org.apache.ignite.internal.index.ColumnCollation;
-import org.apache.ignite.internal.index.Index;
-import org.apache.ignite.internal.index.SortedIndex;
-import org.apache.ignite.internal.index.SortedIndexDescriptor;
-import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
-import org.jetbrains.annotations.Nullable;
-import org.jetbrains.annotations.TestOnly;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelCollation;
+import org.apache.calcite.rel.RelCollations;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.util.ImmutableBitSet;
+import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
+import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
/**
- * Schema object representing an Index.
+ * Auxiliary data structure to represent a table index.
*/
public class IgniteIndex {
/**
@@ -61,130 +58,66 @@ public class IgniteIndex {
}
}
- /**
- * Type of the index.
- */
- public enum Type {
- HASH, SORTED
- }
+ private final int indexId;
- private final List<String> columns;
+ private final String name;
- private final @Nullable List<Collation> collations;
+ private final IgniteDistribution tableDistribution;
- private final Index<?> index;
+ private final RelCollation collation;
private final Type type;
- /**
- * Constructs the Index object.
- *
- * @param index A data access object to wrap.
- */
- public IgniteIndex(Index<?> index) {
- this.index = Objects.requireNonNull(index, "index");
-
- this.columns = index.descriptor().columns();
- this.collations = deriveCollations(index);
- this.type = index instanceof SortedIndex ? Type.SORTED : Type.HASH;
- }
-
- /**
- * Constructs the Index object.
- */
- @TestOnly
- public IgniteIndex(Type type, List<String> columns, @Nullable List<Collation> collations) {
- assert type == Type.SORTED ^ collations == null;
-
- this.columns = columns;
- this.collations = collations;
+ /** Constructor. */
+ public IgniteIndex(int indexId, String name, Type type, IgniteDistribution tableDistribution, RelCollation collation) {
+ this.indexId = indexId;
+ this.name = name;
this.type = type;
-
- index = null;
- }
-
- /** Returns a list of names of indexed columns. */
- public List<String> columns() {
- return columns;
- }
-
- /**
- * Returns a list of collations.
- *
- * <p>The size of the collations list is guaranteed to match the size of indexed columns. The i-th
- * collation is related to an i-th column.
- *
- * @return The list of collations or {@code null} if not applicable.
- */
- public @Nullable List<Collation> collations() {
- return collations;
+ this.tableDistribution = tableDistribution;
+ this.collation = collation;
}
- /** Returns the name of a current index. */
+ /** Returns the name of this index. */
public String name() {
- return index.name();
+ return name;
}
- /** Returns an object providing access to a data. */
- public Index<?> index() {
- return index;
+ /** Returns the type of this index. */
+ public Type type() {
+ return type;
}
- /** Returns id of this index. */
+ /** Returns index id. */
public int id() {
- return index.id();
- }
-
- public int tableId() {
- return index.tableId();
+ return indexId;
}
- public Type type() {
- return type;
+ /** Returns the collation of this index. */
+ public RelCollation collation() {
+ return collation;
}
- //TODO: cache rowType as it can't be changed.
-
- /** Returns index row type.
- *
- * <p>This is a struct type whose fields describe the names and types of indexed columns.</p>
- *
- * <p>The implementer must use the type factory provided. This ensures that
- * the type is converted into a canonical form; other equal types in the same
- * query will use the same object.</p>
- *
- * @param typeFactory Type factory with which to create the type
- * @param tableDescriptor Table descriptor.
- * @return Row type.
+ /**
+ * Translates this index into relational operator.
*/
- public RelDataType getRowType(IgniteTypeFactory typeFactory, TableDescriptor tableDescriptor) {
- RelDataTypeFactory.Builder b = new RelDataTypeFactory.Builder(typeFactory);
-
- for (String colName : columns) {
- ColumnDescriptor colDesc = tableDescriptor.columnDescriptor(colName);
- b.add(colName, native2relationalType(typeFactory, colDesc.physicalType(), colDesc.nullable()));
- }
-
- return b.build();
+ public IgniteLogicalIndexScan toRel(
+ RelOptCluster cluster,
+ RelOptTable relOptTable,
+ List<RexNode> proj,
+ RexNode condition,
+ ImmutableBitSet requiredCols
+ ) {
+ RelTraitSet traitSet = cluster.traitSetOf(Convention.Impl.NONE)
+ .replace(tableDistribution)
+ .replace(type() == Type.HASH ? RelCollations.EMPTY : collation);
+
+ return IgniteLogicalIndexScan.create(cluster, traitSet, relOptTable, name, proj, condition, requiredCols);
}
- private static @Nullable List<Collation> deriveCollations(Index<?> index) {
- if (index.descriptor() instanceof SortedIndexDescriptor) {
- SortedIndexDescriptor descriptor = (SortedIndexDescriptor) index.descriptor();
-
- List<Collation> orders = new ArrayList<>(descriptor.columns().size());
-
- for (var column : descriptor.columns()) {
- ColumnCollation collation = descriptor.collation(column);
-
- assert collation != null;
-
- orders.add(Collation.of(collation.asc(), collation.nullsFirst()));
- }
-
- return List.copyOf(orders);
- }
-
- return null;
+ /**
+ * Type of the index.
+ */
+ public enum Type {
+ HASH, SORTED
}
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaIndex.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaIndex.java
deleted file mode 100644
index 5a5171bca4..0000000000
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaIndex.java
+++ /dev/null
@@ -1,143 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.internal.sql.engine.schema;
-
-import java.util.ArrayList;
-import java.util.List;
-import org.apache.calcite.plan.Convention;
-import org.apache.calcite.plan.RelOptCluster;
-import org.apache.calcite.plan.RelOptTable;
-import org.apache.calcite.plan.RelTraitSet;
-import org.apache.calcite.rel.RelCollation;
-import org.apache.calcite.rel.RelCollations;
-import org.apache.calcite.rel.RelFieldCollation;
-import org.apache.calcite.rel.RelFieldCollation.Direction;
-import org.apache.calcite.rel.RelFieldCollation.NullDirection;
-import org.apache.calcite.rex.RexNode;
-import org.apache.calcite.util.ImmutableBitSet;
-import org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
-import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
-import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
-import org.apache.ignite.internal.catalog.descriptors.CatalogSortedIndexDescriptor;
-import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
-import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
-
-/**
- * Auxiliary data structure to represent a table index.
- */
-public class IgniteSchemaIndex {
-
- private final String name;
-
- private final IgniteDistribution tableDistribution;
-
- private final RelCollation collation;
-
- private final Type type;
-
- /** Constructor. */
- public IgniteSchemaIndex(String name, Type type, IgniteDistribution tableDistribution, RelCollation collation) {
- this.name = name;
- this.type = type;
- this.tableDistribution = tableDistribution;
- this.collation = collation;
- }
-
- /** Returns the name of this index. */
- public String name() {
- return name;
- }
-
- /** Returns the type of this index. */
- public Type type() {
- return type;
- }
-
- /** Returns the collation of this index. */
- public RelCollation collation() {
- return collation;
- }
-
- /**
- * Translates this index into relational operator.
- */
- public IgniteLogicalIndexScan toRel(
- RelOptCluster cluster,
- RelOptTable relOptTable,
- List<RexNode> proj,
- RexNode condition,
- ImmutableBitSet requiredCols
- ) {
- RelTraitSet traitSet = cluster.traitSetOf(Convention.Impl.NONE)
- .replace(tableDistribution)
- .replace(type() == Type.HASH ? RelCollations.EMPTY : collation);
-
- return IgniteLogicalIndexScan.create(cluster, traitSet, relOptTable, name, proj, condition, requiredCols);
- }
-
- static RelCollation createIndexCollation(CatalogIndexDescriptor descriptor, TableDescriptor tableDescriptor) {
- if (descriptor instanceof CatalogSortedIndexDescriptor) {
- CatalogSortedIndexDescriptor sortedIndexDescriptor = (CatalogSortedIndexDescriptor) descriptor;
- List<CatalogIndexColumnDescriptor> columns = sortedIndexDescriptor.columns();
- List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
-
- for (int i = 0; i < columns.size(); i++) {
- CatalogIndexColumnDescriptor column = columns.get(i);
- ColumnDescriptor columnDesc = tableDescriptor.columnDescriptor(column.name());
- int fieldIndex = columnDesc.logicalIndex();
-
- RelFieldCollation fieldCollation;
- switch (column.collation()) {
- case ASC_NULLS_FIRST:
- fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.FIRST);
- break;
- case ASC_NULLS_LAST:
- fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.LAST);
- break;
- case DESC_NULLS_FIRST:
- fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.FIRST);
- break;
- case DESC_NULLS_LAST:
- fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.LAST);
- break;
- default:
- throw new IllegalArgumentException("Unexpected collation: " + column.collation());
- }
-
- fieldCollations.add(fieldCollation);
- }
-
- return RelCollations.of(fieldCollations);
- } else if (descriptor instanceof CatalogHashIndexDescriptor) {
- CatalogHashIndexDescriptor hashIndexDescriptor = (CatalogHashIndexDescriptor) descriptor;
- List<String> columns = hashIndexDescriptor.columns();
- List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
-
- for (String columnName : columns) {
- ColumnDescriptor columnDesc = tableDescriptor.columnDescriptor(columnName);
-
- fieldCollations.add(new RelFieldCollation(columnDesc.logicalIndex(), Direction.CLUSTERED, NullDirection.UNSPECIFIED));
- }
-
- return RelCollations.of(fieldCollations);
- } else {
- throw new IllegalArgumentException("Unexpected index type: " + descriptor);
- }
- }
-}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
index e6d34557e7..367953d654 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteSchemaTable.java
@@ -17,8 +17,11 @@
package org.apache.ignite.internal.sql.engine.schema;
+import java.util.Collections;
import java.util.List;
import java.util.Map;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
import org.apache.calcite.config.CalciteConnectionConfig;
import org.apache.calcite.plan.RelOptCluster;
import org.apache.calcite.plan.RelOptTable;
@@ -35,11 +38,13 @@ import org.apache.calcite.sql.SqlCall;
import org.apache.calcite.sql.SqlNode;
import org.apache.calcite.util.ImmutableBitSet;
import org.apache.ignite.internal.sql.engine.metadata.ColocationGroup;
+import org.apache.ignite.internal.sql.engine.metadata.NodeWithTerm;
import org.apache.ignite.internal.sql.engine.prepare.MappingQueryContext;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
+import org.apache.ignite.internal.table.InternalTable;
import org.jetbrains.annotations.Nullable;
/**
@@ -57,11 +62,11 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
private final IgniteStatistic statistic;
- private final Map<String, IgniteSchemaIndex> indexMap;
+ private final Map<String, IgniteIndex> indexMap;
/** Constructor. */
public IgniteSchemaTable(String name, int tableId, int version, TableDescriptor desc,
- IgniteStatistic statistic, Map<String, IgniteSchemaIndex> indexMap) {
+ IgniteStatistic statistic, Map<String, IgniteIndex> indexMap) {
this.id = tableId;
this.name = name;
@@ -71,6 +76,18 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
this.indexMap = indexMap;
}
+ // TODO: should be moved to a separate component after https://issues.apache.org/jira/browse/IGNITE-18453
+ static Supplier<ColocationGroup> partitionedGroup(InternalTable table) {
+ return () -> {
+ List<List<NodeWithTerm>> assignments = table.primaryReplicas().stream()
+ .map(primaryReplica -> new NodeWithTerm(primaryReplica.node().name(), primaryReplica.term()))
+ .map(Collections::singletonList)
+ .collect(Collectors.toList());
+
+ return ColocationGroup.forAssignments(assignments);
+ };
+ }
+
/** {@inheritDoc} */
@Override
public int id() {
@@ -193,13 +210,13 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
/** {@inheritDoc} */
@Override
public ColocationGroup colocationGroup(MappingQueryContext ctx) {
- return IgniteTableImpl.partitionedGroup(ctx.tableManager().getTable(id).internalTable()).get();
+ return partitionedGroup(ctx.tableManager().getTable(id).internalTable()).get();
}
/** {@inheritDoc} */
@Override
public Map<String, IgniteIndex> indexes() {
- throw new UnsupportedOperationException("getIndexes() should be used instead");
+ return indexMap;
}
/** {@inheritDoc} */
@@ -211,7 +228,7 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
/** {@inheritDoc} */
@Override
public IgniteIndex getIndex(String idxName) {
- throw new UnsupportedOperationException("getIndexes(name) should be used instead");
+ return indexMap.get(idxName);
}
/** {@inheritDoc} */
@@ -219,9 +236,4 @@ public final class IgniteSchemaTable extends AbstractTable implements IgniteTabl
public void removeIndex(String idxName) {
throw new UnsupportedOperationException("IndexMap is not modifiable");
}
-
- /** Returns a map of indexes defined for this table. */
- public Map<String, IgniteSchemaIndex> getIndexes() {
- return indexMap;
- }
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTableImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTableImpl.java
deleted file mode 100644
index 844c0c7d2e..0000000000
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/IgniteTableImpl.java
+++ /dev/null
@@ -1,307 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.internal.sql.engine.schema;
-
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.concurrent.atomic.AtomicLong;
-import java.util.function.DoubleSupplier;
-import java.util.function.Supplier;
-import java.util.stream.Collectors;
-import org.apache.calcite.plan.Convention;
-import org.apache.calcite.plan.RelOptCluster;
-import org.apache.calcite.plan.RelOptTable;
-import org.apache.calcite.plan.RelTraitSet;
-import org.apache.calcite.rel.RelCollation;
-import org.apache.calcite.rel.RelCollations;
-import org.apache.calcite.rel.hint.RelHint;
-import org.apache.calcite.rel.type.RelDataType;
-import org.apache.calcite.rel.type.RelDataTypeFactory;
-import org.apache.calcite.rex.RexNode;
-import org.apache.calcite.schema.Statistic;
-import org.apache.calcite.schema.impl.AbstractTable;
-import org.apache.calcite.util.ImmutableBitSet;
-import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
-import org.apache.ignite.internal.sql.engine.metadata.ColocationGroup;
-import org.apache.ignite.internal.sql.engine.metadata.NodeWithTerm;
-import org.apache.ignite.internal.sql.engine.prepare.MappingQueryContext;
-import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalIndexScan;
-import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
-import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
-import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
-import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
-import org.apache.ignite.internal.storage.MvPartitionStorage;
-import org.apache.ignite.internal.storage.StorageRebalanceException;
-import org.apache.ignite.internal.table.InternalTable;
-import org.jetbrains.annotations.Nullable;
-
-/**
- * Ignite table implementation.
- */
-public class IgniteTableImpl extends AbstractTable implements IgniteTable {
-
- private final TableDescriptor desc;
-
- private final int ver;
-
- private final int id;
-
- private final String name;
-
- private final Supplier<ColocationGroup> colocationGroup;
-
- private final Statistic statistic;
-
- private final Map<String, IgniteIndex> indexes = new HashMap<>();
-
- /**
- * Constructor.
- *
- * @param desc Table descriptor.
- * @param tableId Table id.
- * @param name Table name.
- */
- IgniteTableImpl(TableDescriptor desc, int tableId, String name, int version,
- Supplier<ColocationGroup> colocationGroup, DoubleSupplier rowCount) {
- this.ver = version;
- this.desc = desc;
- this.id = tableId;
- this.name = name;
- this.colocationGroup = colocationGroup;
- this.statistic = new IgniteStatistic(rowCount, desc.distribution());
- }
-
- private IgniteTableImpl(IgniteTableImpl t) {
- this.desc = t.desc;
- this.ver = t.ver;
- this.id = t.id;
- this.name = t.name;
- this.statistic = t.statistic;
- this.colocationGroup = t.colocationGroup;
- this.indexes.putAll(t.indexes);
- }
-
- public static IgniteTableImpl copyOf(IgniteTableImpl v) {
- return new IgniteTableImpl(v);
- }
-
- /** {@inheritDoc} */
- @Override
- public int id() {
- return id;
- }
-
- /** {@inheritDoc} */
- @Override
- public int version() {
- return ver;
- }
-
- /** {@inheritDoc} */
- @Override
- public String name() {
- return name;
- }
-
- /** {@inheritDoc} */
- @Override
- public RelDataType getRowType(RelDataTypeFactory typeFactory, ImmutableBitSet requiredColumns) {
- return desc.rowType((IgniteTypeFactory) typeFactory, requiredColumns);
- }
-
- /** {@inheritDoc} */
- @Override
- public Statistic getStatistic() {
- return statistic;
- }
-
- /** {@inheritDoc} */
- @Override
- public TableDescriptor descriptor() {
- return desc;
- }
-
- /** {@inheritDoc} */
- @Override
- public IgniteLogicalTableScan toRel(
- RelOptCluster cluster,
- RelOptTable relOptTbl,
- List<RelHint> hints,
- @Nullable List<RexNode> proj,
- @Nullable RexNode cond,
- @Nullable ImmutableBitSet requiredColumns
- ) {
- RelTraitSet traitSet = cluster.traitSetOf(distribution());
-
- return IgniteLogicalTableScan.create(cluster, traitSet, hints, relOptTbl, proj, cond, requiredColumns);
- }
-
- /** {@inheritDoc} */
- @Override
- public IgniteLogicalIndexScan toRel(
- RelOptCluster cluster,
- RelOptTable relOptTable,
- String idxName,
- List<RexNode> proj,
- RexNode condition,
- ImmutableBitSet requiredCols
- ) {
- IgniteIndex index = getIndex(idxName);
-
- RelCollation collation = TraitUtils.createCollation(index.columns(), index.collations(), descriptor());
-
- RelTraitSet traitSet = cluster.traitSetOf(Convention.Impl.NONE)
- .replace(distribution())
- .replace(index.type() == Type.HASH ? RelCollations.EMPTY : collation);
-
- return IgniteLogicalIndexScan.create(cluster, traitSet, relOptTable, idxName, proj, condition, requiredCols);
- }
-
- /** {@inheritDoc} */
- @Override
- public IgniteDistribution distribution() {
- return desc.distribution();
- }
-
- /** {@inheritDoc} */
- @Override
- public ColocationGroup colocationGroup(MappingQueryContext ctx) {
- return partitionedGroup();
- }
-
- /** {@inheritDoc} */
- @Override
- public Map<String, IgniteIndex> indexes() {
- return Collections.unmodifiableMap(indexes);
- }
-
- /** {@inheritDoc} */
- @Override
- public void addIndex(IgniteIndex idxTbl) {
- indexes.put(idxTbl.name(), idxTbl);
- }
-
- /** {@inheritDoc} */
- @Override
- public IgniteIndex getIndex(String idxName) {
- return indexes.get(idxName);
- }
-
- /** {@inheritDoc} */
- @Override
- public void removeIndex(String idxName) {
- indexes.remove(idxName);
- }
-
- /** {@inheritDoc} */
- @Override
- public <C> C unwrap(Class<C> cls) {
- if (cls.isInstance(desc)) {
- return cls.cast(desc);
- }
-
- return super.unwrap(cls);
- }
-
- private ColocationGroup partitionedGroup() {
- return colocationGroup.get();
- }
-
- // TODO: should be moved to a separate component after https://issues.apache.org/jira/browse/IGNITE-18453
- static Supplier<ColocationGroup> partitionedGroup(InternalTable table) {
- return () -> {
- List<List<NodeWithTerm>> assignments = table.primaryReplicas().stream()
- .map(primaryReplica -> new NodeWithTerm(primaryReplica.node().name(), primaryReplica.term()))
- .map(Collections::singletonList)
- .collect(Collectors.toList());
-
- return ColocationGroup.forAssignments(assignments);
- };
- }
-
- static DoubleSupplier rowCountStatistic(InternalTable table) {
- return new RowCountStatistic(table);
- }
-
- private static final class RowCountStatistic implements DoubleSupplier {
- private static final int UPDATE_THRESHOLD = DistributionZoneManager.DEFAULT_PARTITION_COUNT;
-
- private final AtomicLong lastUpd = new AtomicLong();
-
- private volatile long localRowCnt = 0L;
-
- private final InternalTable table;
-
- private RowCountStatistic(InternalTable table) {
- this.table = table;
- }
-
- /** {@inheritDoc} */
- @Override
- // TODO: need to be refactored https://issues.apache.org/jira/browse/IGNITE-19558
- public double getAsDouble() {
- int parts = table.storage().getTableDescriptor().getPartitions();
-
- long partitionsRevisionCounter = 0L;
-
- for (int p = 0; p < parts; ++p) {
- @Nullable MvPartitionStorage part = table.storage().getMvPartition(p);
-
- if (part == null) {
- continue;
- }
-
- long upd = part.lastAppliedIndex();
-
- partitionsRevisionCounter += upd;
- }
-
- long prev = lastUpd.get();
-
- if (partitionsRevisionCounter - prev > UPDATE_THRESHOLD) {
- synchronized (this) {
- if (lastUpd.compareAndSet(prev, partitionsRevisionCounter)) {
- long size = 0L;
-
- for (int p = 0; p < parts; ++p) {
- @Nullable MvPartitionStorage part = table.storage().getMvPartition(p);
-
- if (part == null) {
- continue;
- }
-
- try {
- size += part.rowsCount();
- } catch (StorageRebalanceException ignore) {
- // No-op.
- }
- }
-
- localRowCnt = size;
- }
- }
- }
-
- // Forbid zero result, to prevent zero cost for table and index scans.
- return Math.max(10_000.0, (double) localRowCnt);
- }
- }
-}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/SqlSchemaManagerImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/SqlSchemaManagerImpl.java
deleted file mode 100644
index 4a333cca22..0000000000
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/schema/SqlSchemaManagerImpl.java
+++ /dev/null
@@ -1,588 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.internal.sql.engine.schema;
-
-import static java.util.concurrent.CompletableFuture.completedFuture;
-import static java.util.concurrent.CompletableFuture.failedFuture;
-import static org.apache.ignite.internal.sql.engine.SqlQueryProcessor.DEFAULT_SCHEMA_NAME;
-import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
-import static org.apache.ignite.lang.ErrorGroups.Common.INTERNAL_ERR;
-import static org.apache.ignite.lang.ErrorGroups.Common.NODE_STOPPING_ERR;
-import static org.apache.ignite.lang.IgniteStringFormatter.format;
-
-import it.unimi.dsi.fastutil.ints.IntArrayList;
-import it.unimi.dsi.fastutil.ints.IntList;
-import java.util.Comparator;
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
-import java.util.concurrent.CompletableFuture;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.CopyOnWriteArraySet;
-import java.util.function.Consumer;
-import java.util.function.DoubleSupplier;
-import java.util.function.LongFunction;
-import java.util.function.Supplier;
-import java.util.stream.Collectors;
-import org.apache.calcite.schema.SchemaPlus;
-import org.apache.calcite.tools.Frameworks;
-import org.apache.ignite.internal.causality.CompletableVersionedValue;
-import org.apache.ignite.internal.causality.IncrementalVersionedValue;
-import org.apache.ignite.internal.causality.OutdatedTokenException;
-import org.apache.ignite.internal.index.HashIndex;
-import org.apache.ignite.internal.index.Index;
-import org.apache.ignite.internal.index.IndexDescriptor;
-import org.apache.ignite.internal.index.SortedIndexDescriptor;
-import org.apache.ignite.internal.index.SortedIndexImpl;
-import org.apache.ignite.internal.schema.Column;
-import org.apache.ignite.internal.schema.DefaultValueProvider;
-import org.apache.ignite.internal.schema.DefaultValueProvider.Type;
-import org.apache.ignite.internal.schema.SchemaDescriptor;
-import org.apache.ignite.internal.schema.SchemaManager;
-import org.apache.ignite.internal.schema.SchemaRegistry;
-import org.apache.ignite.internal.sql.engine.metadata.ColocationGroup;
-import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
-import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
-import org.apache.ignite.internal.table.InternalTable;
-import org.apache.ignite.internal.table.TableImpl;
-import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.internal.util.IgniteSpinBusyLock;
-import org.apache.ignite.lang.IgniteInternalException;
-import org.apache.ignite.lang.NodeStoppingException;
-import org.jetbrains.annotations.Nullable;
-
-/**
- * Holds actual schema and mutates it on schema change, requested by Ignite.
- */
-public class SqlSchemaManagerImpl implements SqlSchemaManager {
- private final IncrementalVersionedValue<Map<String, IgniteSchema>> schemasVv;
-
- private final IncrementalVersionedValue<Map<Integer, IgniteTable>> tablesVv;
-
- private final Map<Integer, CompletableFuture<?>> pkIdxReady = new ConcurrentHashMap<>();
-
- private final IncrementalVersionedValue<Map<Integer, IgniteIndex>> indicesVv;
-
- private final TableManager tableManager;
- private final SchemaManager schemaManager;
-
- private final CompletableVersionedValue<SchemaPlus> calciteSchemaVv;
-
- private final Set<SchemaUpdateListener> listeners = new CopyOnWriteArraySet<>();
-
- /** Busy lock for stop synchronisation. */
- private final IgniteSpinBusyLock busyLock;
-
- /**
- * Constructor.
- * TODO Documentation https://issues.apache.org/jira/browse/IGNITE-15859
- */
- public SqlSchemaManagerImpl(
- TableManager tableManager,
- SchemaManager schemaManager,
- Consumer<LongFunction<CompletableFuture<?>>> registry,
- IgniteSpinBusyLock busyLock
- ) {
- this.tableManager = tableManager;
- this.schemaManager = schemaManager;
-
- schemasVv = new IncrementalVersionedValue<>(registry, HashMap::new);
- tablesVv = new IncrementalVersionedValue<>(registry, HashMap::new);
- indicesVv = new IncrementalVersionedValue<>(registry, HashMap::new);
- this.busyLock = busyLock;
-
- calciteSchemaVv = new CompletableVersionedValue<>(() -> {
- SchemaPlus newCalciteSchema = Frameworks.createRootSchema(false);
- newCalciteSchema.add(DEFAULT_SCHEMA_NAME, new IgniteSchema(DEFAULT_SCHEMA_NAME));
- return newCalciteSchema;
- });
-
- schemasVv.whenComplete((token, stringIgniteSchemaMap, throwable) -> {
- if (!busyLock.enterBusy()) {
- calciteSchemaVv.completeExceptionally(token, new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
-
- return;
- }
- try {
- if (throwable != null) {
- calciteSchemaVv.completeExceptionally(
- token,
- new IgniteInternalException(
- INTERNAL_ERR, "Couldn't evaluate sql schemas for causality token: " + token, throwable)
- );
-
- return;
- }
-
- SchemaPlus newCalciteSchema = rebuild(stringIgniteSchemaMap);
-
- listeners.forEach(SchemaUpdateListener::onSchemaUpdated);
-
- calciteSchemaVv.complete(token, newCalciteSchema);
- } finally {
- busyLock.leaveBusy();
- }
- });
- }
-
- /** {@inheritDoc} */
- @Override
- public SchemaPlus schema(@Nullable String schema) {
- // stub for waiting pk indexes, more clear place is IgniteSchema
- CompletableFuture.allOf(pkIdxReady.values().toArray(CompletableFuture[]::new)).join();
-
- SchemaPlus schemaPlus = calciteSchemaVv.latest();
-
- return schema != null ? schemaPlus.getSubSchema(schema) : schemaPlus.getSubSchema(DEFAULT_SCHEMA_NAME);
- }
-
- /** {@inheritDoc} */
- @Override
- public SchemaPlus schema(String name, int version) {
- throw new UnsupportedOperationException();
- }
-
- /** {@inheritDoc} */
- @Override
- public CompletableFuture<?> actualSchemaAsync(long ver) {
- if (!busyLock.enterBusy()) {
- throw new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException());
- }
- try {
- if (ver == IgniteSchema.INITIAL_VERSION) {
- return completedFuture(calciteSchemaVv.latest());
- }
-
- CompletableFuture<SchemaPlus> lastSchemaFut;
-
- try {
- lastSchemaFut = calciteSchemaVv.get(ver);
- } catch (OutdatedTokenException e) {
- return completedFuture(null);
- }
-
- return lastSchemaFut;
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- /** {@inheritDoc} */
- @Override
- public SchemaPlus activeSchema(@Nullable String name, long timestamp) {
- throw new UnsupportedOperationException();
- }
-
- /** {@inheritDoc} */
- @Override
- public IgniteTable tableById(int id) {
- if (!busyLock.enterBusy()) {
- throw new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException());
- }
- try {
- IgniteTable table = tablesVv.latest().get(id);
-
- if (table == null) {
- throw new IgniteInternalException(INTERNAL_ERR,
- format("Table not found [tableId={}]", id));
- }
-
- return table;
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- public void registerListener(SchemaUpdateListener listener) {
- listeners.add(listener);
- }
-
- /**
- * OnSqlTypeCreated.
- * TODO Documentation https://issues.apache.org/jira/browse/IGNITE-15859
- */
- public CompletableFuture<?> onTableCreated(String schemaName, int tableId, long causalityToken) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
- }
-
- try {
- pkIdxReady.computeIfAbsent(tableId, k -> new CompletableFuture<>());
-
- CompletableFuture<Map<Integer, IgniteTable>> updatedTables = tablesVv.update(causalityToken, (tables, e) ->
- inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return tableManager.tableAsync(causalityToken, tableId)
- .thenCompose(table -> convert(causalityToken, table))
- .thenApply(igniteTable -> {
- Map<Integer, IgniteTable> resTbls = new HashMap<>(tables);
-
- IgniteTable oldTable = resTbls.put(igniteTable.id(), igniteTable);
-
- // looks like this is UPDATE operation
- if (oldTable != null) {
- for (var index : oldTable.indexes().values()) {
- igniteTable.addIndex(index);
- }
- }
-
- return resTbls;
- });
- })
- );
-
- schemasVv.update(causalityToken, (schemas, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return updatedTables.thenApply(tables -> {
- IgniteTable igniteTable = tables.get(tableId);
-
- Map<String, IgniteSchema> res = new HashMap<>(schemas);
-
- IgniteSchema schema = res.compute(schemaName,
- (k, v) -> v == null ? new IgniteSchema(schemaName, causalityToken) : IgniteSchema.copy(v, causalityToken));
-
- schema.addTable(igniteTable);
-
- return res;
- });
- }));
-
- // calciteSchemaVv depends on all other Versioned Values and is completed last.
- return calciteSchemaVv.get(causalityToken);
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- /**
- * OnSqlTypeUpdated.
- * TODO Documentation https://issues.apache.org/jira/browse/IGNITE-15859
- */
- public CompletableFuture<?> onTableUpdated(String schemaName, int tableId, long causalityToken) {
- return onTableCreated(schemaName, tableId, causalityToken);
- }
-
- /**
- * OnSqlTypeDropped.
- * TODO Documentation https://issues.apache.org/jira/browse/IGNITE-15859
- */
- public CompletableFuture<?> onTableDropped(String schemaName, int tableId, long causalityToken) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
- }
-
- try {
- CompletableFuture<IgniteTable> removedTableFuture = new CompletableFuture<>();
-
- tablesVv.update(causalityToken, (tables, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- removedTableFuture.completeExceptionally(e);
-
- return failedFuture(e);
- }
-
- Map<Integer, IgniteTable> resTbls = new HashMap<>(tables);
-
- IgniteTable removedTable = resTbls.remove(tableId);
-
- removedTableFuture.complete(removedTable);
-
- return completedFuture(resTbls);
- }));
-
- schemasVv.update(causalityToken, (schemas, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return removedTableFuture.thenApply(table -> {
- if (table == null) {
- return schemas;
- }
-
- Map<String, IgniteSchema> res = new HashMap<>(schemas);
-
- IgniteSchema schema = res.compute(schemaName,
- (k, v) -> v == null ? new IgniteSchema(schemaName, causalityToken) : IgniteSchema.copy(v, causalityToken));
-
- schema.removeTable(table.name());
-
- pkIdxReady.remove(table.id());
-
- return res;
- });
- }));
-
- // calciteSchemaVv depends on all other Versioned Values and is completed last.
- return calciteSchemaVv.get(causalityToken);
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- /**
- * Rebuilds Calcite schemas.
- *
- * @param schemas Ignite schemas.
- */
- private SchemaPlus rebuild(Map<String, IgniteSchema> schemas) {
- SchemaPlus newCalciteSchema = Frameworks.createRootSchema(false);
-
- newCalciteSchema.add(DEFAULT_SCHEMA_NAME, new IgniteSchema(DEFAULT_SCHEMA_NAME));
-
- schemas.forEach(newCalciteSchema::add);
-
- return newCalciteSchema;
- }
-
- private CompletableFuture<IgniteTableImpl> convert(long causalityToken, TableImpl table) {
- return schemaManager.schemaRegistry(causalityToken, table.tableId())
- .thenApply(schemaRegistry -> inBusyLock(busyLock, () -> convert(table, schemaRegistry, causalityToken)));
- }
-
- private IgniteTableImpl convert(TableImpl table, SchemaRegistry schemaRegistry, long schemaVersion) {
- SchemaDescriptor descriptor = schemaRegistry.schema();
-
- List<ColumnDescriptor> colDescriptors = descriptor.columnNames().stream()
- .map(descriptor::column)
- .sorted(Comparator.comparingInt(Column::columnOrder))
- .map(col -> new ColumnDescriptorImpl(
- col.name(),
- descriptor.isKeyColumn(col.schemaIndex()),
- col.nullable(),
- col.columnOrder(),
- col.schemaIndex(),
- col.type(),
- convertDefaultValueProvider(col.defaultValueProvider()),
- col::defaultValue
- ))
- .collect(Collectors.toList());
-
- IntList colocationColumns = new IntArrayList();
-
- for (Column column : descriptor.colocationColumns()) {
- colocationColumns.add(column.columnOrder());
- }
-
- // TODO Use the actual zone ID after implementing https://issues.apache.org/jira/browse/IGNITE-18426.
- IgniteDistribution distribution = IgniteDistributions.affinity(colocationColumns, table.tableId(), table.tableId());
-
- InternalTable internalTable = table.internalTable();
- Supplier<ColocationGroup> colocationGroup = IgniteTableImpl.partitionedGroup(internalTable);
- DoubleSupplier rowCount = IgniteTableImpl.rowCountStatistic(internalTable);
-
- return new IgniteTableImpl(
- new TableDescriptorImpl(colDescriptors, distribution),
- internalTable.tableId(),
- internalTable.name(),
- schemaRegistry.lastSchemaVersion(),
- colocationGroup,
- rowCount
- );
- }
-
- private DefaultValueStrategy convertDefaultValueProvider(DefaultValueProvider defaultValueProvider) {
- return defaultValueProvider.type() == Type.CONSTANT
- ? DefaultValueStrategy.DEFAULT_CONSTANT
- : DefaultValueStrategy.DEFAULT_COMPUTED;
- }
-
- /**
- * Index created callback method register index in Calcite schema.
- *
- * @param tableId Table ID.
- * @param indexId Index ID.
- * @param causalityToken Causality token.
- * @return Schema registration future.
- */
- public CompletableFuture<?> onIndexCreated(int tableId, int indexId, IndexDescriptor indexDescriptor, long causalityToken) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
- }
-
- try {
- CompletableFuture<Map<Integer, IgniteIndex>> updatedIndices = indicesVv.update(causalityToken, (indices, e) ->
- inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return tableManager.tableAsync(causalityToken, tableId).thenApply(table -> {
- var igniteIndex = new IgniteIndex(newIndex(table, indexId, indexDescriptor));
-
- Map<Integer, IgniteIndex> resIdxs = new HashMap<>(indices);
-
- resIdxs.put(indexId, igniteIndex);
-
- return resIdxs;
- });
- }));
-
- CompletableFuture<Map<Integer, IgniteTable>> updatedTables = tablesVv.update(causalityToken, (tables, e) ->
- inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return updatedIndices.thenApply(indices -> {
- IgniteIndex igniteIndex = indices.get(indexId);
-
- Map<Integer, IgniteTable> resTbls = new HashMap<>(tables);
-
- IgniteTable igniteTable = resTbls.computeIfPresent(tableId,
- (k, v) -> IgniteTableImpl.copyOf((IgniteTableImpl) v));
-
- assert igniteTable != null : "Table " + tableId + " was not found";
-
- igniteTable.addIndex(igniteIndex);
-
- return resTbls;
- });
- }));
-
- schemasVv.update(causalityToken, (schemas, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- return updatedTables.thenCombine(updatedIndices, (tables, indices) -> inBusyLock(busyLock, () -> {
- Map<String, IgniteSchema> res = new HashMap<>(schemas);
-
- IgniteSchema schema = res.compute(DEFAULT_SCHEMA_NAME,
- (k, v) -> v == null ? new IgniteSchema(k, causalityToken) : IgniteSchema.copy(v, causalityToken));
-
- schema.addTable(tables.get(tableId));
-
- schema.addIndex(indexId, indices.get(indexId));
-
- return res;
- }));
- }));
-
- // this stub is necessary for observing pk index creation.
- schemasVv.whenComplete((token, stringIgniteSchemaMap, throwable) -> {
- CompletableFuture<?> pkFut = pkIdxReady.get(tableId);
- // this listener is called repeatedly on node stop.
- if (pkFut != null) {
- pkFut.complete(null);
- }
- });
-
- return calciteSchemaVv.get(causalityToken);
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- private static Index<?> newIndex(TableImpl table, int indexId, IndexDescriptor descriptor) {
- if (descriptor instanceof SortedIndexDescriptor) {
- return new SortedIndexImpl(indexId, table.internalTable(), (SortedIndexDescriptor) descriptor);
- } else {
- return new HashIndex(indexId, table.internalTable(), descriptor);
- }
- }
-
- /**
- * Index dropped callback method deregisters index from Calcite schema.
- *
- * @param schemaName Schema name.
- * @param indexId Index id.
- * @param causalityToken Causality token.
- * @return Schema registration future.
- */
- public CompletableFuture<?> onIndexDropped(String schemaName, int tableId, int indexId, long causalityToken) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new IgniteInternalException(NODE_STOPPING_ERR, new NodeStoppingException()));
- }
-
- try {
- CompletableFuture<IgniteIndex> removedIndexFuture = new CompletableFuture<>();
-
- indicesVv.update(causalityToken, (indices, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- removedIndexFuture.completeExceptionally(e);
-
- return failedFuture(e);
- }
-
- Map<Integer, IgniteIndex> resIdxs = new HashMap<>(indices);
-
- IgniteIndex rmvIdx = resIdxs.remove(indexId);
-
- removedIndexFuture.complete(rmvIdx);
-
- return completedFuture(resIdxs);
- }));
-
- CompletableFuture<Map<Integer, IgniteTable>> updatedTables = tablesVv.update(causalityToken, (tables, e) ->
- inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- Map<Integer, IgniteTable> resTbls = new HashMap<>(tables);
-
- IgniteTable table = resTbls.computeIfPresent(tableId, (k, v) -> IgniteTableImpl.copyOf((IgniteTableImpl) v));
-
- if (table == null) {
- return completedFuture(resTbls);
- } else {
- return removedIndexFuture.thenApply(rmvIndex -> {
- table.removeIndex(rmvIndex.name());
-
- return resTbls;
- });
- }
- }));
-
- schemasVv.update(causalityToken, (schemas, e) -> inBusyLock(busyLock, () -> {
- if (e != null) {
- return failedFuture(e);
- }
-
- Map<String, IgniteSchema> res = new HashMap<>(schemas);
-
- IgniteSchema schema = res.compute(schemaName,
- (k, v) -> v == null ? new IgniteSchema(schemaName, causalityToken) : IgniteSchema.copy(v, causalityToken));
-
- schema.removeIndex(indexId);
-
- return updatedTables.thenApply(tables -> {
- IgniteTable table = tables.get(tableId);
-
- if (table != null) {
- schema.addTable(tables.get(tableId));
- }
-
- return res;
- });
- }));
-
- return calciteSchemaVv.get(causalityToken);
- } finally {
- busyLock.leaveBusy();
- }
- }
-}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/TraitUtils.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/TraitUtils.java
index 858a6380b5..8357b5a1f6 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/TraitUtils.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/trait/TraitUtils.java
@@ -24,7 +24,6 @@ import static org.apache.calcite.rel.RelDistribution.Type.BROADCAST_DISTRIBUTED;
import static org.apache.calcite.rel.RelDistribution.Type.HASH_DISTRIBUTED;
import static org.apache.ignite.internal.util.CollectionUtils.first;
import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
-import static org.apache.ignite.lang.IgniteStringFormatter.format;
import com.google.common.collect.ImmutableList;
import it.unimi.dsi.fastutil.ints.Int2IntOpenHashMap;
@@ -58,6 +57,10 @@ import org.apache.calcite.rex.RexSlot;
import org.apache.calcite.util.ControlFlowException;
import org.apache.calcite.util.Pair;
import org.apache.calcite.util.mapping.Mappings;
+import org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogSortedIndexDescriptor;
import org.apache.ignite.internal.index.ColumnCollation;
import org.apache.ignite.internal.sql.engine.rel.IgniteConvention;
import org.apache.ignite.internal.sql.engine.rel.IgniteExchange;
@@ -65,11 +68,7 @@ import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.rel.IgniteSort;
import org.apache.ignite.internal.sql.engine.rel.IgniteTrimExchange;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Collation;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
-import org.apache.ignite.lang.ErrorGroups.Common;
-import org.apache.ignite.lang.IgniteInternalException;
import org.jetbrains.annotations.Nullable;
/**
@@ -411,56 +410,60 @@ public class TraitUtils {
}
/**
- * Creates {@link RelCollation} object from a given collations.
+ * Creates {@link RelCollation} object for given index.
*
- * @param collations List of collations.
+ * @param descriptor Index descriptor.
+ * @param tableDescriptor Table descriptor to derive column indexes from.
* @return a {@link RelCollation} object.
*/
- public static RelCollation createCollation(List<Collation> collations) {
- List<RelFieldCollation> fieldCollations = new ArrayList<>(collations.size());
+ public static RelCollation createIndexCollation(CatalogIndexDescriptor descriptor, TableDescriptor tableDescriptor) {
+ if (descriptor instanceof CatalogSortedIndexDescriptor) {
+ CatalogSortedIndexDescriptor sortedIndexDescriptor = (CatalogSortedIndexDescriptor) descriptor;
+ List<CatalogIndexColumnDescriptor> columns = sortedIndexDescriptor.columns();
+ List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
+
+ for (int i = 0; i < columns.size(); i++) {
+ CatalogIndexColumnDescriptor column = columns.get(i);
+ ColumnDescriptor columnDesc = tableDescriptor.columnDescriptor(column.name());
+ int fieldIndex = columnDesc.logicalIndex();
+
+ RelFieldCollation fieldCollation;
+ switch (column.collation()) {
+ case ASC_NULLS_FIRST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.FIRST);
+ break;
+ case ASC_NULLS_LAST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.LAST);
+ break;
+ case DESC_NULLS_FIRST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.FIRST);
+ break;
+ case DESC_NULLS_LAST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.LAST);
+ break;
+ default:
+ throw new IllegalArgumentException("Unexpected collation: " + column.collation());
+ }
- for (int i = 0; i < collations.size(); i++) {
- fieldCollations.add(createFieldCollation(i, collations.get(i)));
- }
+ fieldCollations.add(fieldCollation);
+ }
- return RelCollations.of(fieldCollations);
- }
+ return RelCollations.of(fieldCollations);
+ } else if (descriptor instanceof CatalogHashIndexDescriptor) {
+ CatalogHashIndexDescriptor hashIndexDescriptor = (CatalogHashIndexDescriptor) descriptor;
+ List<String> columns = hashIndexDescriptor.columns();
+ List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
- /**
- * Creates {@link RelCollation} object from a given collations.
- *
- * @param indexedColumns List of columns names.
- * @param collations List of collations.
- * @param descriptor Table descriptor to derive column indexes from.
- * @return a {@link RelCollation} object.
- */
- public static RelCollation createCollation(
- List<String> indexedColumns,
- @Nullable List<IgniteIndex.Collation> collations,
- TableDescriptor descriptor
- ) {
- List<RelFieldCollation> fieldCollations = new ArrayList<>(indexedColumns.size());
-
- if (collations == null) { // Build collation for Hash index.
- for (int i = 0; i < indexedColumns.size(); i++) {
- String columnName = indexedColumns.get(i);
- ColumnDescriptor columnDesc = descriptor.columnDescriptor(columnName);
+ for (String columnName : columns) {
+ ColumnDescriptor columnDesc = tableDescriptor.columnDescriptor(columnName);
fieldCollations.add(new RelFieldCollation(columnDesc.logicalIndex(), Direction.CLUSTERED, NullDirection.UNSPECIFIED));
}
return RelCollations.of(fieldCollations);
+ } else {
+ throw new IllegalArgumentException("Unexpected index type: " + descriptor);
}
-
- for (int i = 0; i < indexedColumns.size(); i++) {
- String columnName = indexedColumns.get(i);
- IgniteIndex.Collation collation = collations.get(i);
- ColumnDescriptor columnDesc = descriptor.columnDescriptor(columnName);
-
- fieldCollations.add(createFieldCollation(columnDesc.logicalIndex(), collation));
- }
-
- return RelCollations.of(fieldCollations);
}
/**
@@ -485,22 +488,6 @@ public class TraitUtils {
return new RelFieldCollation(fieldIdx, direction, nullDirection);
}
- /** Creates field collation. */
- public static RelFieldCollation createFieldCollation(int fieldIdx, IgniteIndex.Collation collation) {
- switch (collation) {
- case ASC_NULLS_LAST:
- return new RelFieldCollation(fieldIdx, RelFieldCollation.Direction.ASCENDING, RelFieldCollation.NullDirection.LAST);
- case ASC_NULLS_FIRST:
- return new RelFieldCollation(fieldIdx, RelFieldCollation.Direction.ASCENDING, RelFieldCollation.NullDirection.FIRST);
- case DESC_NULLS_LAST:
- return new RelFieldCollation(fieldIdx, RelFieldCollation.Direction.DESCENDING, RelFieldCollation.NullDirection.LAST);
- case DESC_NULLS_FIRST:
- return new RelFieldCollation(fieldIdx, RelFieldCollation.Direction.DESCENDING, RelFieldCollation.NullDirection.FIRST);
- default:
- throw new IgniteInternalException(Common.INTERNAL_ERR, format("Unknown collation [collation={}]", collation));
- }
- }
-
public static boolean distributionEnabled(RelNode node) {
return distribution(node) != null;
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
index 041738819a..6a73dff178 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ExecutionDependencyResolverSelfTest.java
@@ -29,17 +29,18 @@ import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.HashMap;
-import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
+import org.apache.calcite.rel.RelCollations;
import org.apache.calcite.rel.type.RelDataType;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.sql.type.SqlTypeName;
import org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest;
import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
@@ -114,7 +115,9 @@ public class ExecutionDependencyResolverSelfTest extends AbstractPlannerTest {
int t1Id = tester.addTable("TEST1", tableType);
tester.setDependencies(t1Id, table1, update1);
- tester.addIndex("TEST1", new IgniteIndex(TestHashIndex.create(List.of("ID"), "ID_IDX")));
+ tester.addIndex("TEST1", new IgniteIndex(1, "ID_IDX", Type.HASH,
+ tester.tableDescriptor("TEST1").distribution(),
+ RelCollations.of(0)));
CompletableFuture<ResolvedDependencies> f = tester.resolveDependencies("SELECT * FROM test1 WHERE id=1");
tester.checkDependencies(f.join(), t1Id);
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNodeExecutionTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNodeExecutionTest.java
index 1b6fb59cde..02e84724a1 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNodeExecutionTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/rel/IndexScanNodeExecutionTest.java
@@ -36,12 +36,11 @@ import org.apache.calcite.rel.type.RelDataType;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.rel.type.RelDataTypeFactory.Builder;
import org.apache.calcite.sql.type.SqlTypeName;
-import org.apache.ignite.internal.index.ColumnCollation;
-import org.apache.ignite.internal.index.HashIndex;
-import org.apache.ignite.internal.index.Index;
-import org.apache.ignite.internal.index.IndexDescriptor;
-import org.apache.ignite.internal.index.SortedIndexDescriptor;
-import org.apache.ignite.internal.index.SortedIndexImpl;
+import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
+import org.apache.ignite.internal.catalog.descriptors.CatalogHashIndexDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexColumnDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogSortedIndexDescriptor;
import org.apache.ignite.internal.sql.engine.exec.ExecutionContext;
import org.apache.ignite.internal.sql.engine.exec.RowHandler.RowFactory;
import org.apache.ignite.internal.sql.engine.exec.ScannableTable;
@@ -51,11 +50,10 @@ import org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestTab
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
+import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
import org.apache.ignite.internal.sql.engine.util.Commons;
-import org.apache.ignite.internal.table.InternalTable;
import org.jetbrains.annotations.Nullable;
import org.junit.jupiter.api.Test;
-import org.mockito.Mockito;
/**
* Test {@link IndexScanNode} execution.
@@ -67,7 +65,7 @@ public class IndexScanNodeExecutionTest extends AbstractExecutionTest {
*/
@Test
public void testSortedIndex() {
- Index<?> index = newSortedIndex();
+ CatalogSortedIndexDescriptor index = newSortedIndex();
ExecutionContext<Object[]> ctx = executionContext();
@@ -90,7 +88,7 @@ public class IndexScanNodeExecutionTest extends AbstractExecutionTest {
*/
@Test
public void testHashIndex() {
- Index<?> index = newHashIndex();
+ CatalogHashIndexDescriptor index = newHashIndex();
ExecutionContext<Object[]> ctx = executionContext();
@@ -114,11 +112,12 @@ public class IndexScanNodeExecutionTest extends AbstractExecutionTest {
this.ctx = ctx;
}
- IndexScanNode<Object[]> createSortedIndex(Index<?> index, TestScannableTable<?> scannableTable, Comparator<Object[]> cmp) {
+ IndexScanNode<Object[]> createSortedIndex(CatalogIndexDescriptor index, TestScannableTable<?> scannableTable,
+ Comparator<Object[]> cmp) {
return createIndexNode(ctx, index, scannableTable, cmp);
}
- IndexScanNode<Object[]> createHashIndex(Index<?> index, TestScannableTable<?> scannableTable) {
+ IndexScanNode<Object[]> createHashIndex(CatalogIndexDescriptor index, TestScannableTable<?> scannableTable) {
return createIndexNode(ctx, index, scannableTable, null);
}
@@ -150,39 +149,48 @@ public class IndexScanNodeExecutionTest extends AbstractExecutionTest {
}
}
- private Index<?> newHashIndex() {
- IndexDescriptor descriptor = new IndexDescriptor("IDX", List.of("C1"));
+ private CatalogHashIndexDescriptor newHashIndex() {
+ return new CatalogHashIndexDescriptor(1, "IDX", 0, false, List.of("C1"));
- return new HashIndex(1, Mockito.mock(InternalTable.class), descriptor);
}
- private Index<?> newSortedIndex() {
- List<String> columnNames = List.of("C1");
- List<ColumnCollation> columnCollations = List.of(ColumnCollation.ASC_NULLS_LAST);
- SortedIndexDescriptor descriptor = new SortedIndexDescriptor("IDX", columnNames, columnCollations);
-
- return new SortedIndexImpl(1, Mockito.mock(InternalTable.class), descriptor);
+ private CatalogSortedIndexDescriptor newSortedIndex() {
+ return new CatalogSortedIndexDescriptor(1, "IDX", 0, false,
+ List.of(new CatalogIndexColumnDescriptor("C1", CatalogColumnCollation.ASC_NULLS_LAST)));
}
- private IndexScanNode<Object[]> createIndexNode(ExecutionContext<Object[]> ctx, Index<?> index,
+ private IndexScanNode<Object[]> createIndexNode(ExecutionContext<Object[]> ctx, CatalogIndexDescriptor indexDescriptor,
TestScannableTable<?> scannableTable, @Nullable Comparator<Object[]> comparator) {
RelDataTypeFactory.Builder rowTypeBuilder = new Builder(Commons.typeFactory());
- for (String column : index.descriptor().columns()) {
- rowTypeBuilder = rowTypeBuilder.add(column, SqlTypeName.INTEGER);
+ if (indexDescriptor instanceof CatalogHashIndexDescriptor) {
+ for (String column : ((CatalogHashIndexDescriptor) indexDescriptor).columns()) {
+ rowTypeBuilder = rowTypeBuilder.add(column, SqlTypeName.INTEGER);
+ }
+ } else {
+ for (CatalogIndexColumnDescriptor column : ((CatalogSortedIndexDescriptor) indexDescriptor).columns()) {
+ rowTypeBuilder = rowTypeBuilder.add(column.name(), SqlTypeName.INTEGER);
+ }
}
RelDataType rowType = rowTypeBuilder.build();
TableDescriptor tableDescriptor = new TestTableDescriptor(IgniteDistributions::single, rowType);
- IgniteIndex schemaIndex = new IgniteIndex(index);
+ IgniteIndex schemaIndex = new IgniteIndex(
+ indexDescriptor.id(),
+ indexDescriptor.name(),
+ indexDescriptor instanceof CatalogHashIndexDescriptor ? IgniteIndex.Type.HASH : IgniteIndex.Type.SORTED,
+ tableDescriptor.distribution(),
+ TraitUtils.createIndexCollation(indexDescriptor, tableDescriptor)
+ );
+
RowFactory<Object[]> rowFactory = ctx.rowHandler().factory(ctx.getTypeFactory(), rowType);
SingleRangeIterable<Object[]> conditions = new SingleRangeIterable<>(new Object[]{}, null, false, false);
List<PartitionWithTerm> partitions = scannableTable.getPartitions();
- return new IndexScanNode<>(ctx, rowFactory, schemaIndex, scannableTable, tableDescriptor, partitions,
+ return new IndexScanNode<>(indexDescriptor.id(), ctx, rowFactory, schemaIndex, scannableTable, tableDescriptor, partitions,
comparator, conditions, null, null, null);
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/schema/SqlSchemaManagerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/schema/SqlSchemaManagerTest.java
deleted file mode 100644
index 6061fbf5d0..0000000000
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/schema/SqlSchemaManagerTest.java
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.internal.sql.engine.exec.schema;
-
-import static java.util.concurrent.CompletableFuture.allOf;
-import static java.util.concurrent.CompletableFuture.completedFuture;
-import static org.apache.ignite.internal.testframework.IgniteTestUtils.await;
-import static org.junit.jupiter.api.Assertions.assertEquals;
-import static org.junit.jupiter.api.Assertions.assertInstanceOf;
-import static org.junit.jupiter.api.Assertions.assertNotNull;
-import static org.junit.jupiter.api.Assertions.assertNotSame;
-import static org.junit.jupiter.api.Assertions.assertNull;
-import static org.junit.jupiter.api.Assertions.assertSame;
-import static org.mockito.ArgumentMatchers.anyInt;
-import static org.mockito.ArgumentMatchers.anyLong;
-import static org.mockito.ArgumentMatchers.eq;
-import static org.mockito.Mockito.mock;
-import static org.mockito.Mockito.verifyNoMoreInteractions;
-import static org.mockito.Mockito.when;
-
-import java.util.List;
-import java.util.concurrent.CompletableFuture;
-import java.util.concurrent.atomic.AtomicLong;
-import java.util.function.Consumer;
-import java.util.function.LongFunction;
-import org.apache.calcite.schema.SchemaPlus;
-import org.apache.calcite.schema.Table;
-import org.apache.ignite.internal.index.Index;
-import org.apache.ignite.internal.index.IndexDescriptor;
-import org.apache.ignite.internal.schema.Column;
-import org.apache.ignite.internal.schema.NativeTypes;
-import org.apache.ignite.internal.schema.SchemaDescriptor;
-import org.apache.ignite.internal.schema.SchemaManager;
-import org.apache.ignite.internal.schema.registry.SchemaRegistryImpl;
-import org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestHashIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
-import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
-import org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl;
-import org.apache.ignite.internal.sql.engine.schema.SqlSchemaManagerImpl;
-import org.apache.ignite.internal.table.InternalTable;
-import org.apache.ignite.internal.table.TableImpl;
-import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.internal.util.IgniteSpinBusyLock;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Test;
-import org.junit.jupiter.api.extension.ExtendWith;
-import org.mockito.Mock;
-import org.mockito.Mockito;
-import org.mockito.junit.jupiter.MockitoExtension;
-
-/**
- * Tests to verify {@link SqlSchemaManagerImpl}.
- */
-@ExtendWith(MockitoExtension.class)
-public class SqlSchemaManagerTest {
- private final int tableId = 1;
-
- private final int indexId = 2;
-
- private final SchemaDescriptor schemaDescriptor = new SchemaDescriptor(
- 1,
- new Column[]{new Column(0, "ID", NativeTypes.INT64, false)},
- new Column[]{new Column(1, "VAL", NativeTypes.INT64, false)}
- );
-
- @Mock
- private TableManager tableManager;
-
- @Mock
- private SchemaManager schemaManager;
-
- @Mock
- private TableImpl table;
-
- @Mock
- private Index<IndexDescriptor> index;
-
- @Mock
- private SchemaRegistryImpl schemaRegistry;
-
- private SqlSchemaManagerImpl sqlSchemaManager;
-
- private TestRevisionRegister testRevisionRegister;
-
- /** Busy lock for stop synchronisation. */
- private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
-
- @BeforeEach
- public void setup() {
- Mockito.reset(tableManager);
-
- when(tableManager.tableAsync(anyLong(), eq(tableId))).thenReturn(completedFuture(table));
-
- testRevisionRegister = new TestRevisionRegister();
-
- sqlSchemaManager = new SqlSchemaManagerImpl(
- tableManager,
- schemaManager,
- testRevisionRegister,
- busyLock
- );
-
- testRevisionRegister.moveForward();
- }
-
- @Test
- public void testOnTableDroppedHandler() {
- InternalTable mock = mock(InternalTable.class);
- when(mock.tableId()).thenReturn(tableId);
- when(mock.name()).thenReturn("T");
-
- when(table.internalTable()).thenReturn(mock);
- when(table.tableId()).thenReturn(tableId);
- when(schemaRegistry.schema()).thenReturn(schemaDescriptor);
- when(schemaRegistry.lastSchemaVersion()).thenReturn(schemaDescriptor.version());
-
- when(schemaManager.schemaRegistry(anyLong(), anyInt())).thenReturn(completedFuture(schemaRegistry));
-
- sqlSchemaManager.onTableCreated("PUBLIC", tableId, testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- TestHashIndex testHashIndex = TestHashIndex.create(List.of("ID"), "pk_idx", tableId);
-
- sqlSchemaManager.onIndexCreated(
- testHashIndex.tableId(),
- testHashIndex.id(),
- testHashIndex.descriptor(),
- testRevisionRegister.actualToken() + 1
- );
-
- testRevisionRegister.moveForward();
-
- Table schemaTable = sqlSchemaManager.schema("PUBLIC").getTable("T");
-
- assertNotNull(schemaTable);
- IgniteTableImpl igniteTable = assertInstanceOf(IgniteTableImpl.class, schemaTable);
- assertEquals(tableId, igniteTable.id());
-
- sqlSchemaManager.onTableDropped("PUBLIC", tableId, testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- assertNull(sqlSchemaManager.schema("PUBLIC").getTable("T"));
- }
-
- @Test
- public void testIndexEventHandler() {
- InternalTable mock = mock(InternalTable.class);
- when(mock.tableId()).thenReturn(tableId);
- when(mock.name()).thenReturn("T");
-
- when(table.internalTable()).thenReturn(mock);
- when(table.tableId()).thenReturn(tableId);
- when(schemaRegistry.schema()).thenReturn(schemaDescriptor);
- when(schemaRegistry.lastSchemaVersion()).thenReturn(schemaDescriptor.version());
- when(schemaManager.schemaRegistry(anyLong(), anyInt())).thenReturn(completedFuture(schemaRegistry));
-
- sqlSchemaManager.onTableCreated("PUBLIC", tableId, testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- TestHashIndex testHashIndex = TestHashIndex.create(List.of("ID"), "pk_idx", tableId);
-
- sqlSchemaManager.onIndexCreated(
- testHashIndex.tableId(),
- testHashIndex.id(),
- testHashIndex.descriptor(),
- testRevisionRegister.actualToken() + 1
- );
-
- testRevisionRegister.moveForward();
-
- assertEquals(1, ((IgniteTableImpl) sqlSchemaManager.schema("PUBLIC").getTable("T")).indexes().size());
-
- IndexDescriptor descMock = mock(IndexDescriptor.class);
- when(descMock.columns()).thenReturn(List.of());
- when(descMock.name()).thenReturn("PUBLIC.I");
-
- sqlSchemaManager.onIndexCreated(tableId, indexId, descMock, testRevisionRegister.actualToken() + 1);
-
- testRevisionRegister.moveForward();
-
- IgniteSchema schema = sqlSchemaManager.schema("PUBLIC").unwrap(IgniteSchema.class);
- Table schemaTable = schema.getTable("T");
- IgniteIndex igniteIndex = schema.index(indexId);
-
- assertNotNull(igniteIndex);
-
- IgniteTableImpl igniteTable = assertInstanceOf(IgniteTableImpl.class, schemaTable);
-
- assertEquals(igniteTable.id(), igniteIndex.tableId());
- assertSame(igniteIndex, igniteTable.indexes().get("PUBLIC.I"));
-
- sqlSchemaManager.onIndexDropped("PUBLIC", igniteTable.id(), indexId, testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- assertNull(sqlSchemaManager.schema("PUBLIC").unwrap(IgniteSchema.class).index(indexId));
-
- verifyNoMoreInteractions(tableManager);
- }
-
-
- @Test
- public void testIndexEventsProcessed() {
- InternalTable mock = mock(InternalTable.class);
- when(mock.tableId()).thenReturn(tableId);
- when(mock.name()).thenReturn("T");
-
- when(table.internalTable()).thenReturn(mock);
- when(table.tableId()).thenReturn(tableId);
- when(schemaRegistry.schema()).thenReturn(schemaDescriptor);
- when(schemaRegistry.lastSchemaVersion()).thenReturn(schemaDescriptor.version());
- when(schemaManager.schemaRegistry(anyLong(), anyInt())).thenReturn(completedFuture(schemaRegistry));
-
- sqlSchemaManager.onTableCreated("PUBLIC", table.tableId(), testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- TestHashIndex testHashIndex = TestHashIndex.create(List.of("ID"), "pk_idx", tableId);
-
- sqlSchemaManager.onIndexCreated(
- testHashIndex.tableId(),
- testHashIndex.id(),
- testHashIndex.descriptor(),
- testRevisionRegister.actualToken() + 1
- );
-
- testRevisionRegister.moveForward();
-
- String idxName = "I";
-
- IndexDescriptor descMock = mock(IndexDescriptor.class);
- when(descMock.columns()).thenReturn(List.of());
- when(descMock.name()).thenReturn(idxName);
-
- {
- SchemaPlus schema1 = sqlSchemaManager.schema("PUBLIC");
-
- sqlSchemaManager.onIndexCreated(tableId, indexId, descMock, testRevisionRegister.actualToken() + 1);
- testRevisionRegister.moveForward();
-
- SchemaPlus schema2 = sqlSchemaManager.schema("PUBLIC");
-
- // Validate schema snapshot.
- assertNotSame(schema1, schema2);
- assertNotSame(schema1.getTable("T"), schema2.getTable("T"));
-
- assertNull(schema1.unwrap(IgniteSchema.class).index(indexId));
- assertNotNull(schema2.unwrap(IgniteSchema.class).index(indexId));
-
- assertNull(((IgniteTable) schema1.getTable("T")).getIndex(idxName));
- assertNotNull(((IgniteTable) schema2.getTable("T")).getIndex(idxName));
- }
- {
- sqlSchemaManager.onIndexDropped("PUBLIC", table.tableId(), indexId, testRevisionRegister.actualToken() + 1);
- SchemaPlus schema1 = sqlSchemaManager.schema("PUBLIC");
- testRevisionRegister.moveForward();
-
- SchemaPlus schema2 = sqlSchemaManager.schema("PUBLIC");
-
- // Validate schema snapshot.
- assertNotSame(schema1, schema2);
- assertNotSame(schema1.getTable("T"), schema2.getTable("T"));
-
- assertNotNull(schema1.unwrap(IgniteSchema.class).index(indexId));
- assertNull(schema2.unwrap(IgniteSchema.class).index(indexId));
-
- assertNull(((IgniteTable) schema2.getTable("T")).getIndex(idxName));
- assertNotNull(((IgniteTable) schema1.getTable("T")).getIndex(idxName));
- }
-
- verifyNoMoreInteractions(tableManager);
- }
-
- /**
- * Test revision register.
- */
- private static class TestRevisionRegister implements Consumer<LongFunction<CompletableFuture<?>>> {
- AtomicLong token = new AtomicLong(-1);
-
- /** Revision consumer. */
- private LongFunction<CompletableFuture<?>> moveRevision;
-
- /**
- * Moves forward token.
- */
- void moveForward() {
- await(moveRevision.apply(token.incrementAndGet()));
- }
-
- /**
- * Gets an actual token.
- *
- * @return Actual token.
- */
- long actualToken() {
- return token.get();
- }
-
- /** {@inheritDoc} */
- @Override
- public void accept(LongFunction<CompletableFuture<?>> function) {
- if (moveRevision == null) {
- moveRevision = function;
- } else {
- LongFunction<CompletableFuture<?>> old = moveRevision;
-
- moveRevision = rev -> allOf(
- old.apply(rev),
- function.apply(rev)
- );
- }
- }
- }
-}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestBuilders.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestBuilders.java
index 1673721150..f705763bb0 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestBuilders.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestBuilders.java
@@ -31,7 +31,13 @@ import java.util.Set;
import java.util.UUID;
import java.util.function.Function;
import java.util.stream.Collectors;
+import org.apache.calcite.rel.RelCollation;
+import org.apache.calcite.rel.RelCollations;
+import org.apache.calcite.rel.RelFieldCollation;
+import org.apache.calcite.rel.RelFieldCollation.Direction;
+import org.apache.calcite.rel.RelFieldCollation.NullDirection;
import org.apache.calcite.schema.Table;
+import org.apache.ignite.internal.catalog.descriptors.CatalogColumnCollation;
import org.apache.ignite.internal.schema.NativeType;
import org.apache.ignite.internal.sql.engine.exec.ArrayRowHandler;
import org.apache.ignite.internal.sql.engine.exec.ExecutionContext;
@@ -42,9 +48,9 @@ import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptorImpl;
import org.apache.ignite.internal.sql.engine.schema.DefaultValueStrategy;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Collation;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
+import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptorImpl;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
import org.apache.ignite.internal.sql.engine.util.BaseQueryContext;
@@ -431,7 +437,7 @@ public class TestBuilders {
size
);
- indexBuilders.stream().map(AbstractIndexBuilderImpl::build).forEach(testTable::addIndex);
+ indexBuilders.stream().map(idxBuilder -> idxBuilder.build(testTable)).forEach(testTable::addIndex);
return testTable;
}
@@ -479,7 +485,7 @@ public class TestBuilders {
private TestTable build() {
TestTable testTable = new TestTable(new TableDescriptorImpl(columns, distribution), name, dataProviders, size);
- indexBuilders.forEach(idx -> testTable.addIndex(idx.build()));
+ indexBuilders.forEach(idx -> testTable.addIndex(idx.build(testTable)));
return testTable;
}
@@ -509,7 +515,7 @@ public class TestBuilders {
/** {@inheritDoc} */
@Override
- public TestIndex build() {
+ public TestIndex build(TestTable table) {
if (name == null) {
throw new IllegalArgumentException("Name is not specified");
}
@@ -522,7 +528,9 @@ public class TestBuilders {
throw new IllegalArgumentException("Collation must be specified for each of columns.");
}
- return new TestIndex(name, Type.SORTED, columns, collations, dataProviders);
+ RelCollation relCollation = sortedIndexCollation(columns, collations, table.descriptor());
+
+ return new TestIndex(name, Type.SORTED, table.distribution(), relCollation, dataProviders);
}
}
@@ -549,7 +557,7 @@ public class TestBuilders {
/** {@inheritDoc} */
@Override
- public TestIndex build() {
+ public TestIndex build(TestTable table) {
if (name == null) {
throw new IllegalArgumentException("Name is not specified");
}
@@ -560,7 +568,10 @@ public class TestBuilders {
assert collations == null : "Collation is not supported.";
- return new TestIndex(name, Type.HASH, columns, null, dataProviders);
+
+ RelCollation relCollation = hashIndexCollation(columns, table.descriptor());
+
+ return new TestIndex(name, Type.SORTED, table.distribution(), relCollation, dataProviders);
}
}
@@ -587,10 +598,12 @@ public class TestBuilders {
}
@Override
- TestIndex build() {
+ TestIndex build(TestTable table) {
assert collations.size() == columns.size();
- return TestIndex.createSorted(name, columns, collations, dataProviders);
+ RelCollation relCollation = sortedIndexCollation(columns, collations, table.descriptor());
+
+ return new TestIndex(name, Type.SORTED, table.distribution(), relCollation, dataProviders);
}
}
@@ -617,10 +630,12 @@ public class TestBuilders {
}
@Override
- TestIndex build() {
+ TestIndex build(TestTable table) {
assert collations == null;
- return TestIndex.createHash(name, columns, dataProviders);
+ RelCollation relCollation = hashIndexCollation(columns, table.descriptor());
+
+ return new TestIndex(name, Type.SORTED, table.distribution(), relCollation, dataProviders);
}
}
@@ -700,7 +715,7 @@ public class TestBuilders {
private abstract static class AbstractIndexBuilderImpl<ChildT> extends AbstractDataSourceBuilderImpl<ChildT>
implements SortedIndexBuilderBase<ChildT>, HashIndexBuilderBase<ChildT> {
protected final List<String> columns = new ArrayList<>();
- protected List<Collation> collations;
+ protected List<CatalogColumnCollation> collations;
/** {@inheritDoc} */
@Override
@@ -712,7 +727,7 @@ public class TestBuilders {
/** {@inheritDoc} */
@Override
- public ChildT addColumn(String columnName, Collation collation) {
+ public ChildT addColumn(String columnName, CatalogColumnCollation collation) {
if (collations == null) {
collations = new ArrayList<>();
}
@@ -723,7 +738,7 @@ public class TestBuilders {
return self();
}
- abstract TestIndex build();
+ abstract TestIndex build(TestTable table);
}
private abstract static class AbstractDataSourceBuilderImpl<ChildT> {
@@ -802,7 +817,7 @@ public class TestBuilders {
*/
private interface SortedIndexBuilderBase<ChildT> extends IndexBuilderBase<ChildT> {
/** Adds a column with specified collation to the index. */
- ChildT addColumn(String columnName, Collation collation);
+ ChildT addColumn(String columnName, CatalogColumnCollation collation);
}
/**
@@ -848,4 +863,48 @@ public class TestBuilders {
*/
ParentT end();
}
+
+ private static RelCollation sortedIndexCollation(List<String> columns, List<CatalogColumnCollation> collations, TableDescriptor table) {
+ assert collations.size() == columns.size();
+
+ List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
+ for (int i = 0; i < columns.size(); i++) {
+ ColumnDescriptor columnDesc = table.columnDescriptor(columns.get(i));
+ int fieldIndex = columnDesc.logicalIndex();
+
+ RelFieldCollation fieldCollation;
+ switch (collations.get(i)) {
+ case ASC_NULLS_FIRST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.FIRST);
+ break;
+ case ASC_NULLS_LAST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.ASCENDING, NullDirection.LAST);
+ break;
+ case DESC_NULLS_FIRST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.FIRST);
+ break;
+ case DESC_NULLS_LAST:
+ fieldCollation = new RelFieldCollation(fieldIndex, Direction.DESCENDING, NullDirection.LAST);
+ break;
+ default:
+ throw new IllegalArgumentException("Unexpected collation: " + collations.get(i));
+ }
+
+ fieldCollations.add(fieldCollation);
+ }
+
+ return RelCollations.of(fieldCollations);
+ }
+
+ private static RelCollation hashIndexCollation(List<String> columns, TableDescriptor tableDescriptor) {
+ List<RelFieldCollation> fieldCollations = new ArrayList<>(columns.size());
+
+ for (String columnName : columns) {
+ ColumnDescriptor columnDesc = tableDescriptor.columnDescriptor(columnName);
+
+ fieldCollations.add(new RelFieldCollation(columnDesc.logicalIndex(), Direction.CLUSTERED, NullDirection.UNSPECIFIED));
+ }
+
+ return RelCollations.of(fieldCollations);
+ }
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestIndex.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestIndex.java
index f9da45677b..aaae987d4a 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestIndex.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/framework/TestIndex.java
@@ -19,11 +19,11 @@ package org.apache.ignite.internal.sql.engine.framework;
import static org.apache.ignite.lang.IgniteStringFormatter.format;
-import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
+import org.apache.calcite.rel.RelCollation;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
-import org.jetbrains.annotations.Nullable;
+import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
/**
* A test index that implements all the necessary for the optimizer methods to be used to prepare a query, as well as provides access to the
@@ -34,48 +34,34 @@ public class TestIndex extends IgniteIndex {
"DataProvider is not configured [index={}, node={}]";
/** Factory method for creating hash-index. */
- static TestIndex createHash(String name, List<String> indexedColumns, Map<String, DataProvider<?>> dataProviders) {
- return new TestIndex(name, Type.HASH, indexedColumns, null, dataProviders);
+ static TestIndex createHash(String name, IgniteDistribution distribution, RelCollation collations,
+ Map<String, DataProvider<?>> dataProviders) {
+ return new TestIndex(name, Type.HASH, distribution, collations, dataProviders);
}
/** Factory method for creating sorted-index. */
- static TestIndex createSorted(String name, List<String> columns, List<Collation> collations,
+ static TestIndex createSorted(String name, IgniteDistribution distribution, RelCollation collations,
Map<String, DataProvider<?>> dataProviders) {
- return new TestIndex(name, Type.SORTED, columns, collations, dataProviders);
+ return new TestIndex(name, Type.SORTED, distribution, collations, dataProviders);
}
private static final AtomicInteger ID = new AtomicInteger();
- private final int id = ID.incrementAndGet();
- private final String name;
-
private final Map<String, DataProvider<?>> dataProviders;
/** Constructor. */
TestIndex(
String name,
Type type,
- List<String> columns,
- @Nullable List<Collation> collations,
+ IgniteDistribution distribution,
+ RelCollation collations,
Map<String, DataProvider<?>> dataProviders
) {
- super(type, columns, collations);
+ super(ID.incrementAndGet(), name, type, distribution, collations);
- this.name = name;
this.dataProviders = dataProviders;
}
- /** Returns an id of the index. */
- public int id() {
- return id;
- }
-
- /** {@inheritDoc} */
- @Override
- public String name() {
- return name;
- }
-
/**
* Returns the data provider for the given node.
*
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractAggregatePlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractAggregatePlannerTest.java
index fbff6b9e03..f231d66dbc 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractAggregatePlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractAggregatePlannerTest.java
@@ -39,6 +39,7 @@ import org.apache.ignite.internal.sql.engine.framework.TestBuilders;
import org.apache.ignite.internal.sql.engine.rel.IgniteAggregate;
import org.apache.ignite.internal.sql.engine.rel.agg.IgniteReduceAggregateBase;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
import org.apache.ignite.internal.sql.engine.trait.TraitUtils;
@@ -556,7 +557,7 @@ public abstract class AbstractAggregatePlannerTest extends AbstractPlannerTest {
private static IgniteIndex createIndex(org.apache.ignite.internal.sql.engine.framework.TestTable tbl, String name,
RelCollation collation) {
- return new IgniteIndex(TestSortedIndex.create(collation, name, tbl));
+ return new IgniteIndex(1, name, Type.SORTED, tbl.distribution(), collation);
}
<T extends RelNode> Predicate<T> hasAggregate() {
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractPlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractPlannerTest.java
index d4ea5e47b1..6e7f88ad18 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractPlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/AbstractPlannerTest.java
@@ -114,6 +114,7 @@ import org.apache.ignite.internal.sql.engine.rel.logical.IgniteLogicalTableScan;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
import org.apache.ignite.internal.sql.engine.schema.DefaultValueStrategy;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.schema.IgniteTable;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
@@ -1002,7 +1003,7 @@ public abstract class AbstractPlannerTest extends IgniteAbstractTest {
* TODO Documentation https://issues.apache.org/jira/browse/IGNITE-15859
*/
public TestTable addIndex(RelCollation collation, String name) {
- indexes.put(name, new IgniteIndex(TestSortedIndex.create(collation, name, this)));
+ indexes.put(name, new IgniteIndex(1, name, Type.SORTED, distribution(), collation));
return this;
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/HashIndexPlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/HashIndexPlannerTest.java
index c84fa99d77..2c6081575f 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/HashIndexPlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/HashIndexPlannerTest.java
@@ -22,14 +22,12 @@ import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.notNullValue;
import static org.hamcrest.Matchers.nullValue;
-import java.util.List;
import org.apache.calcite.plan.RelOptPlanner.CannotPlanException;
import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.rel.RelNode;
import org.apache.ignite.internal.sql.engine.rel.IgniteIndexScan;
import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
import org.apache.ignite.internal.testframework.IgniteTestUtils;
@@ -113,7 +111,7 @@ public class HashIndexPlannerTest extends AbstractPlannerTest {
"VAL", Integer.class
);
- tbl.addIndex(new IgniteIndex(TestHashIndex.create(List.of("VAL"), indexName)));
+ tbl.addIndex(indexName, 1);
return createSchema(tbl);
}
@@ -138,7 +136,7 @@ public class HashIndexPlannerTest extends AbstractPlannerTest {
"VAL1", Integer.class
);
- rightTable.addIndex(new IgniteIndex(TestHashIndex.create(List.of("VAL0", "VAL1"), indexName)));
+ rightTable.addIndex(indexName, 1, 2);
IgniteSchema schema = createSchema(leftTable, rightTable);
@@ -174,7 +172,7 @@ public class HashIndexPlannerTest extends AbstractPlannerTest {
"VAL1", Integer.class
);
- rightTable.addIndex(new IgniteIndex(TestHashIndex.create(List.of("VAL0", "VAL1"), indexName)));
+ rightTable.addIndex(indexName, 1, 2);
IgniteSchema schema = createSchema(leftTable, rightTable);
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ImplicitCastsTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ImplicitCastsTest.java
index 8f4f1e213f..50de9383be 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ImplicitCastsTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ImplicitCastsTest.java
@@ -44,6 +44,7 @@ import org.apache.ignite.internal.sql.engine.rel.IgniteMergeJoin;
import org.apache.ignite.internal.sql.engine.rel.IgniteNestedLoopJoin;
import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableScan;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
@@ -271,8 +272,8 @@ public class ImplicitCastsTest extends AbstractPlannerTest {
}
}
- Stream<Arguments> s1 = filterTypes().filter(filter).map(new AddIndexType(Type.SORTED));
- Stream<Arguments> s2 = filterTypes().filter(filter).map(new AddIndexType(Type.HASH));
+ Stream<Arguments> s1 = filterTypes().filter(filter).map(new AddIndexType(IgniteIndex.Type.SORTED));
+ Stream<Arguments> s2 = filterTypes().filter(filter).map(new AddIndexType(IgniteIndex.Type.HASH));
return Stream.concat(s1, s2);
}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/JoinColocationPlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/JoinColocationPlannerTest.java
index b8decb7a1f..d49b5be243 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/JoinColocationPlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/JoinColocationPlannerTest.java
@@ -26,7 +26,6 @@ import static org.hamcrest.Matchers.hasSize;
import java.util.List;
import org.apache.calcite.plan.RelOptUtil;
-import org.apache.calcite.rel.RelCollations;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.util.ImmutableIntList;
import org.apache.ignite.internal.sql.engine.rel.IgniteExchange;
@@ -35,7 +34,6 @@ import org.apache.ignite.internal.sql.engine.rel.IgniteMergeJoin;
import org.apache.ignite.internal.sql.engine.rel.IgniteRel;
import org.apache.ignite.internal.sql.engine.rel.IgniteSort;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableScan;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
import org.junit.jupiter.api.Test;
@@ -56,7 +54,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- tbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(0), "PK", tbl)));
+ tbl.addIndex("PK", 0);
IgniteSchema schema = createSchema(tbl);
@@ -89,7 +87,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- tbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(0, 1)), "PK", tbl)));
+ tbl.addIndex("PK", 0, 1);
IgniteSchema schema = createSchema(tbl);
@@ -125,7 +123,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- complexTbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(0, 1)), "PK", complexTbl)));
+ complexTbl.addIndex("PK", 0, 1);
TestTable simpleTbl = createTable(
"SIMPLE_TBL",
@@ -136,7 +134,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- simpleTbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(0), "PK", simpleTbl)));
+ simpleTbl.addIndex("PK", 0);
IgniteSchema schema = createSchema(complexTbl, simpleTbl);
@@ -179,8 +177,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- complexTblDirect.addIndex(new IgniteIndex(
- TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(0, 1)), "PK", complexTblDirect)));
+ complexTblDirect.addIndex("PK", 0, 1);
TestTable complexTblIndirect = createTable(
"COMPLEX_TBL_INDIRECT",
@@ -191,8 +188,7 @@ public class JoinColocationPlannerTest extends AbstractPlannerTest {
"VAL", String.class
);
- complexTblIndirect.addIndex(new IgniteIndex(
- TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(0, 1)), "PK", complexTblIndirect)));
+ complexTblIndirect.addIndex("PK", 0, 1);
IgniteSchema schema = createSchema(complexTblDirect, complexTblIndirect);
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
index 3499c20e0b..25e967974b 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/PlannerTest.java
@@ -31,7 +31,6 @@ import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Predicate;
import org.apache.calcite.plan.RelOptUtil;
import org.apache.calcite.plan.RelTraitSet;
-import org.apache.calcite.rel.RelCollations;
import org.apache.calcite.rel.RelNode;
import org.apache.calcite.rel.RelRoot;
import org.apache.calcite.rel.RelVisitor;
@@ -40,7 +39,6 @@ import org.apache.calcite.rel.hint.HintStrategyTable;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.calcite.sql.SqlNode;
-import org.apache.calcite.util.ImmutableIntList;
import org.apache.calcite.util.Util;
import org.apache.ignite.internal.sql.engine.metadata.ColocationGroup;
import org.apache.ignite.internal.sql.engine.metadata.MappingService;
@@ -767,7 +765,7 @@ public class PlannerTest extends AbstractPlannerTest {
}
};
- emp.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(1, 2)), "emp_idx", emp)));
+ emp.addIndex("emp_idx", 1, 2);
TestTable dept = new TestTable("DEPT",
new RelDataTypeFactory.Builder(f)
@@ -781,7 +779,7 @@ public class PlannerTest extends AbstractPlannerTest {
}
};
- dept.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(ImmutableIntList.of(1, 0)), "dep_idx", dept)));
+ dept.addIndex("dep_idx", 1, 0);
IgniteSchema publicSchema = new IgniteSchema("PUBLIC");
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ProjectFilterScanMergePlannerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ProjectFilterScanMergePlannerTest.java
index 066bc211c0..8674e300b1 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ProjectFilterScanMergePlannerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/planner/ProjectFilterScanMergePlannerTest.java
@@ -31,6 +31,7 @@ import org.apache.ignite.internal.sql.engine.rel.IgniteAggregate;
import org.apache.ignite.internal.sql.engine.rel.IgniteIndexScan;
import org.apache.ignite.internal.sql.engine.rel.IgniteTableScan;
import org.apache.ignite.internal.sql.engine.schema.IgniteIndex;
+import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.schema.IgniteSchema;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
@@ -91,7 +92,7 @@ public class ProjectFilterScanMergePlannerTest extends AbstractPlannerTest {
public void testProjectFilterMergeSortedIndex() throws Exception {
// Test project and filter merge into index scan.
TestTable tbl = ((TestTable) publicSchema.getTable("TBL"));
- tbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(2), "IDX_C", tbl)));
+ tbl.addIndex("IDX_C", 2);
// Without index condition shift.
assertPlan("SELECT a, b FROM tbl WHERE c = 0", publicSchema, isInstanceOf(IgniteIndexScan.class)
@@ -118,7 +119,7 @@ public class ProjectFilterScanMergePlannerTest extends AbstractPlannerTest {
public void testProjectFilterMergeHashIndex() throws Exception {
// Test project and filter merge into index scan.
TestTable tbl = ((TestTable) publicSchema.getTable("TBL"));
- tbl.addIndex(new IgniteIndex(TestHashIndex.create(List.of("c"), "IDX_C")));
+ tbl.addIndex(new IgniteIndex(1, "IDX_C", Type.HASH, tbl.distribution(), RelCollations.of(2)));
// Without index condition shift.
assertPlan("SELECT a, b FROM tbl WHERE c = 0", publicSchema, isInstanceOf(IgniteIndexScan.class)
@@ -145,7 +146,7 @@ public class ProjectFilterScanMergePlannerTest extends AbstractPlannerTest {
public void testIdentityFilterMergeIndex() throws Exception {
// Test project and filter merge into index scan.
TestTable tbl = ((TestTable) publicSchema.getTable("TBL"));
- tbl.addIndex(new IgniteIndex(TestSortedIndex.create(RelCollations.of(2), "IDX_C", tbl)));
+ tbl.addIndex("IDX_C", 2);
// Without index condition shift.
assertPlan("SELECT a, b, c FROM tbl WHERE c = 0", publicSchema, isInstanceOf(IgniteIndexScan.class)
@@ -170,7 +171,7 @@ public class ProjectFilterScanMergePlannerTest extends AbstractPlannerTest {
public void testIdentityFilterMergeHashIndex() throws Exception {
// Test project and filter merge into index scan.
TestTable tbl = ((TestTable) publicSchema.getTable("TBL"));
- tbl.addIndex(new IgniteIndex(TestHashIndex.create(List.of("c"), "IDX_C")));
+ tbl.addIndex(new IgniteIndex(1, "IDX_C", Type.HASH, tbl.distribution(), RelCollations.of(2)));
// Without index condition shift.
assertPlan("SELECT a, b, c FROM tbl WHERE c = 0", publicSchema, isInstanceOf(IgniteIndexScan.class)
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
index aecb8d8eaa..894a6a60ba 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/schema/CatalogSqlSchemaManagerTest.java
@@ -57,7 +57,6 @@ import org.apache.ignite.internal.catalog.descriptors.CatalogTableColumnDescript
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.schema.DefaultValueGenerator;
import org.apache.ignite.internal.schema.NativeType;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Type;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistribution;
import org.apache.ignite.internal.sql.engine.trait.IgniteDistributions;
import org.apache.ignite.internal.sql.engine.util.TypeUtils;
@@ -375,10 +374,10 @@ public class CatalogSqlSchemaManagerTest {
IgniteSchemaTable table = (IgniteSchemaTable) schema.getTable(testTable.name);
assertNotNull(table);
- IgniteSchemaIndex testIdx = table.getIndexes().get(testIndex.name);
+ IgniteIndex testIdx = table.getIndex(testIndex.name);
assertEquals(testIndex.name, testIdx.name());
- assertEquals(Type.HASH, testIdx.type());
+ assertEquals(IgniteIndex.Type.HASH, testIdx.type());
assertEquals(RelCollations.of(
new RelFieldCollation(0, Direction.CLUSTERED, NullDirection.UNSPECIFIED),
new RelFieldCollation(1, Direction.CLUSTERED, NullDirection.UNSPECIFIED)
@@ -419,10 +418,10 @@ public class CatalogSqlSchemaManagerTest {
IgniteSchemaTable table = (IgniteSchemaTable) schema.getTable(testTable.name);
assertNotNull(table);
- IgniteSchemaIndex testIdx = table.getIndexes().get(testIndex.name);
+ IgniteIndex testIdx = table.getIndex(testIndex.name);
assertEquals(testIndex.name, testIdx.name());
- assertEquals(Type.SORTED, testIdx.type());
+ assertEquals(IgniteIndex.Type.SORTED, testIdx.type());
assertEquals(RelCollations.of(
new RelFieldCollation(0, Direction.ASCENDING, NullDirection.LAST),
new RelFieldCollation(1, Direction.ASCENDING, NullDirection.FIRST),
[ignite-3] 17/19: wip. Replace direct proxy with waiting for latest catalog version.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 605b8f6b29234288ebeab347ba1b086037041ddb
Author: amashenkov <an...@gmail.com>
AuthorDate: Mon Jul 24 16:35:48 2023 +0300
wip. Replace direct proxy with waiting for latest catalog version.
---
.../ignite/internal/catalog/CatalogManager.java | 8 +
.../internal/catalog/CatalogManagerImpl.java | 7 +
.../ignite/internal/catalog/storage/UpdateLog.java | 5 +
.../internal/catalog/storage/UpdateLogImpl.java | 9 +-
.../apache/ignite/internal/index/IndexManager.java | 10 +-
.../ignite/internal/schema/SchemaManager.java | 2 -
.../sql/engine/exec/UpdatableTableImpl.java | 2 -
.../internal/table/distributed/TableManager.java | 417 +++------------------
8 files changed, 72 insertions(+), 388 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java
index 1d91cc35de..351aaccbfa 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManager.java
@@ -131,4 +131,12 @@ public interface CatalogManager extends IgniteComponent, CatalogService {
* @return Operation future.
*/
CompletableFuture<Void> renameDistributionZone(RenameZoneParams params);
+
+ // TODO: replace with safe-time waiting.
+ /**
+ * Waits for latest catalog version.
+ *
+ * @return Operation future.
+ */
+ CompletableFuture<Void> waitLatest();
}
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index fe30bbcf89..92407146b4 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -699,6 +699,13 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
throw new SqlException(STMT_VALIDATION_ERR, msg, params);
}
+ /** {@inheritDoc} */
+ @Override
+ public CompletableFuture<Void> waitLatest() {
+ return updateLog.lastUpdateVersion()
+ .thenCompose(versionTracker::waitFor);
+ }
+
@FunctionalInterface
interface UpdateProducer {
List<UpdateEntry> get(Catalog catalog);
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java
index eb17046103..51500d86c5 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLog.java
@@ -58,6 +58,11 @@ public interface UpdateLog extends IgniteComponent {
*/
@Override void start() throws IgniteInternalException;
+ /**
+ * Returns last update version.
+ */
+ CompletableFuture<Integer> lastUpdateVersion();
+
/**
* An interface describing a handler that will receive notification
* when a new update is added to the log.
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
index 4d0e00198c..2bfa223e00 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/storage/UpdateLogImpl.java
@@ -55,7 +55,7 @@ public class UpdateLogImpl implements UpdateLog {
private final IgniteSpinBusyLock busyLock = new IgniteSpinBusyLock();
private final AtomicBoolean stopGuard = new AtomicBoolean();
- private final MetaStorageManager metastore;
+ public final MetaStorageManager metastore;
private volatile OnUpdateHandler onUpdateHandler;
private volatile @Nullable UpdateListener listener;
@@ -170,6 +170,13 @@ public class UpdateLogImpl implements UpdateLog {
}
}
+ /** {@inheritDoc} */
+ @Override
+ public CompletableFuture<Integer> lastUpdateVersion() {
+ return metastore.get(CatalogKey.currentVersion())
+ .thenApply(e -> e.value() == null ? 0 : ByteUtils.bytesToInt(e.value()));
+ }
+
private static class CatalogKey {
private CatalogKey() {
throw new AssertionError();
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index 4a4853c2b8..c9c23d2318 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -17,7 +17,6 @@
package org.apache.ignite.internal.index;
-import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import static org.apache.ignite.internal.util.ArrayUtils.STRING_EMPTY_ARRAY;
@@ -201,15 +200,13 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
try {
fireEvent(IndexEvent.DROP, new IndexEventParameters(causalityToken, tableId, idxId));
- CompletableFuture<?> dropIndexFuture = tableManager.tableAsync(evt.causalityToken(), tableId)
+ return tableManager.tablesById(evt.causalityToken())
+ .thenApply(tables -> tables.get(tableId))
.thenAccept(table -> {
if (table != null) { // in case of DROP TABLE the table will be removed first
table.unregisterIndex(idxId);
}
});
-
- // TODO: investigate why DropIndexFuture hangs.
- return completedFuture(null); // dropIndexFuture;
} catch (Throwable th) {
LOG.warn("Failed to process drop index event.", th);
@@ -267,9 +264,6 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
var storageIndexDescriptor = StorageIndexDescriptor.create(tableDescriptor, indexDescriptor);
- CompletableFuture<?> fireEventFuture =
- fireEvent(IndexEvent.CREATE, new IndexEventParameters(causalityToken, tableId, indexId, eventIndexDescriptor));
-
TableImpl table = tableManager.getTable(tableId);
assert table != null : tableId;
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
index af0451b259..f383a8f940 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/SchemaManager.java
@@ -70,8 +70,6 @@ import org.jetbrains.annotations.Nullable;
* The class services a management of table schemas.
*/
public class SchemaManager extends Producer<SchemaEvent, SchemaEventParameters> implements IgniteComponent {
- private static final IgniteLogger LOGGER = Loggers.forClass(SchemaManager.class);
-
/** Initial version for schemas. */
public static final int INITIAL_SCHEMA_VERSION = 1;
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
index 3f48868249..94503daf75 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/UpdatableTableImpl.java
@@ -26,7 +26,6 @@ import it.unimi.dsi.fastutil.ints.Int2ObjectOpenHashMap;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Collection;
-import java.util.Comparator;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import org.apache.ignite.internal.hlc.HybridClock;
@@ -43,7 +42,6 @@ import org.apache.ignite.internal.schema.row.Row;
import org.apache.ignite.internal.schema.row.RowAssembler;
import org.apache.ignite.internal.sql.engine.metadata.NodeWithTerm;
import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptor;
-import org.apache.ignite.internal.sql.engine.schema.ColumnDescriptorImpl;
import org.apache.ignite.internal.sql.engine.schema.TableDescriptor;
import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.sql.engine.util.TypeUtils;
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index ee926b253d..afbc7093be 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -25,14 +25,11 @@ import static java.util.concurrent.CompletableFuture.failedFuture;
import static java.util.concurrent.CompletableFuture.runAsync;
import static java.util.concurrent.CompletableFuture.supplyAsync;
import static java.util.stream.Collectors.toList;
-import static org.apache.ignite.internal.catalog.CatalogService.DEFAULT_SCHEMA_NAME;
import static org.apache.ignite.internal.causality.IncrementalVersionedValue.dependingOn;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.partitionAssignments;
import static org.apache.ignite.internal.distributionzones.rebalance.RebalanceUtil.tableAssignments;
import static org.apache.ignite.internal.distributionzones.rebalance.ZoneCatalogDescriptorUtils.toZoneDescriptor;
import static org.apache.ignite.internal.metastorage.dsl.Operations.put;
-import static org.apache.ignite.internal.schema.SchemaManager.INITIAL_SCHEMA_VERSION;
-import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
import static org.apache.ignite.internal.util.IgniteUtils.inBusyLock;
import static org.apache.ignite.internal.util.IgniteUtils.shutdownAndAwaitTermination;
import static org.apache.ignite.internal.utils.RebalanceUtil.ASSIGNMENTS_SWITCH_REDUCE_PREFIX;
@@ -57,7 +54,6 @@ import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
-import java.util.NoSuchElementException;
import java.util.Objects;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
@@ -81,8 +77,6 @@ import java.util.function.LongFunction;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.Stream;
-import org.apache.ignite.configuration.ConfigurationChangeException;
-import org.apache.ignite.configuration.ConfigurationProperty;
import org.apache.ignite.internal.affinity.AffinityUtils;
import org.apache.ignite.internal.affinity.Assignment;
import org.apache.ignite.internal.baseline.BaselineManager;
@@ -94,6 +88,7 @@ import org.apache.ignite.internal.catalog.commands.DropTableParams;
import org.apache.ignite.internal.catalog.descriptors.CatalogDataStorageDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogIndexDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogObjectDescriptor;
+import org.apache.ignite.internal.catalog.descriptors.CatalogSchemaDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogTableDescriptor;
import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.catalog.events.CatalogEvent;
@@ -131,17 +126,11 @@ import org.apache.ignite.internal.raft.storage.impl.LogStorageFactoryCreator;
import org.apache.ignite.internal.replicator.ReplicaManager;
import org.apache.ignite.internal.replicator.ReplicaService;
import org.apache.ignite.internal.replicator.TablePartitionId;
-import org.apache.ignite.internal.schema.CatalogDescriptorUtils;
import org.apache.ignite.internal.schema.SchemaManager;
import org.apache.ignite.internal.schema.SchemaRegistry;
-import org.apache.ignite.internal.schema.configuration.ExtendedTableChange;
import org.apache.ignite.internal.schema.configuration.GcConfiguration;
import org.apache.ignite.internal.schema.configuration.TableChange;
-import org.apache.ignite.internal.schema.configuration.TableConfiguration;
-import org.apache.ignite.internal.schema.configuration.TablesChange;
import org.apache.ignite.internal.schema.configuration.TablesConfiguration;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
-import org.apache.ignite.internal.schema.configuration.storage.DataStorageView;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.storage.MvPartitionStorage;
import org.apache.ignite.internal.storage.RowId;
@@ -189,9 +178,7 @@ import org.apache.ignite.internal.util.PendingComparableValuesTracker;
import org.apache.ignite.internal.utils.RebalanceUtil;
import org.apache.ignite.internal.vault.VaultManager;
import org.apache.ignite.lang.ByteArray;
-import org.apache.ignite.lang.DistributionZoneNotFoundException;
import org.apache.ignite.lang.IgniteException;
-import org.apache.ignite.lang.IgniteExceptionUtils;
import org.apache.ignite.lang.IgniteInternalException;
import org.apache.ignite.lang.IgniteSystemProperties;
import org.apache.ignite.lang.NodeStoppingException;
@@ -203,7 +190,6 @@ import org.apache.ignite.network.MessagingService;
import org.apache.ignite.network.TopologyService;
import org.apache.ignite.raft.jraft.storage.impl.VolatileRaftMetaStorage;
import org.apache.ignite.table.Table;
-import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
import org.jetbrains.annotations.TestOnly;
@@ -375,8 +361,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
private final IndexBuilder indexBuilder;
- private final ConfiguredTablesCache configuredTablesCache;
-
/**
* Creates a new table manager.
*
@@ -498,8 +482,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
lowWatermark = new LowWatermark(nodeName, gcConfig.lowWatermark(), clock, txManager, vaultManager, mvGc);
indexBuilder = new IndexBuilder(nodeName, cpus);
-
- configuredTablesCache = new ConfiguredTablesCache(tablesCfg, getMetadataLocallyOnly);
}
@Override
@@ -675,7 +657,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
CatalogTableDescriptor tableDescriptor = catalogManager.table(evt.tableId(), previousCatalogVersion);
CatalogZoneDescriptor zoneDescriptor = catalogManager.zone(tableDescriptor.zoneId(), previousCatalogVersion);
- dropTableLocally(evt.causalityToken(), tableDescriptor, zoneDescriptor);
+ return dropTableLocally(evt.causalityToken(), tableDescriptor, zoneDescriptor);
} catch (Throwable th) {
LOG.warn("Failed to process drop table event.", th);
@@ -683,8 +665,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
} finally {
busyLock.leaveBusy();
}
-
- return completedFuture(null);
}
/**
@@ -1279,15 +1259,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
createPartsFut.thenRunAsync(() -> inBusyLock(busyLock, () -> completeApiCreateFuture(table)));
- // TODO should be reworked in IGNITE-16763
- // We use the event notification future as the result so that dependent components can complete the schema updates.
-
- // TODO: https://issues.apache.org/jira/browse/IGNITE-19913 Possible performance degradation.
- fireEvent(TableEvent.CREATE, new TableEventParameters(causalityToken, tableId));
-
- // TODO: investigate why createParts future hangs.
- // return createPartsFut;
- return completedFuture(false);
+ return createPartsFut;
}
/**
@@ -1370,7 +1342,11 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @param tableDescriptor Catalog table descriptor.
* @param zoneDescriptor Catalog distributed zone descriptor.
*/
- private void dropTableLocally(long causalityToken, CatalogTableDescriptor tableDescriptor, CatalogZoneDescriptor zoneDescriptor) {
+ private CompletableFuture<Void> dropTableLocally(
+ long causalityToken,
+ CatalogTableDescriptor tableDescriptor,
+ CatalogZoneDescriptor zoneDescriptor
+ ) {
int tableId = tableDescriptor.id();
int partitions = zoneDescriptor.partitions();
@@ -1431,22 +1407,19 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
startedTables.remove(tableId);
- fireEvent(TableEvent.DROP, new TableEventParameters(causalityToken, tableId))
- .whenComplete((v, e) -> {
- Set<ByteArray> assignmentKeys = new HashSet<>();
+ Set<ByteArray> assignmentKeys = new HashSet<>();
- for (int p = 0; p < partitions; p++) {
- assignmentKeys.add(stablePartAssignmentsKey(new TablePartitionId(tableId, p)));
- }
+ for (int p = 0; p < partitions; p++) {
+ assignmentKeys.add(stablePartAssignmentsKey(new TablePartitionId(tableId, p)));
+ }
- metaStorageMgr.removeAll(assignmentKeys);
+ metaStorageMgr.removeAll(assignmentKeys);
- if (e != null) {
- LOG.error("Error on " + TableEvent.DROP + " notification", e);
- }
- });
+ return completedFuture(null);
} catch (NodeStoppingException e) {
fireEvent(TableEvent.DROP, new TableEventParameters(causalityToken, tableId), e);
+
+ return failedFuture(e);
}
}
@@ -1474,180 +1447,15 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
String tableName = parameters.tableName();
- String zoneName = parameters.zone();
-
- // Copied from DdlCommandHandler
- Consumer<TableChange> tblChanger = tableChange -> {
- tableChange.changeColumns(columnsChange -> {
- for (var col : parameters.columns()) {
- columnsChange.create(col.name(), columnChange -> CatalogDescriptorUtils.convertColumnDefinition(col, columnChange));
- }
- });
-
- List<String> colocationKeys = nullOrEmpty(parameters.colocationColumns())
- ? parameters.primaryKeyColumns()
- : parameters.colocationColumns();
-
- tableChange.changePrimaryKey(pkChange -> pkChange.changeColumns(parameters.primaryKeyColumns().toArray(String[]::new))
- .changeColocationColumns(colocationKeys.toArray(String[]::new)));
- };
return catalogManager.createTable(parameters)
.thenApply(ignore -> catalogManager.table(tableName, Long.MAX_VALUE).id())
- .thenCompose(tableId -> createTableAsyncInternal(tableId, tableName, zoneName, tblChanger));
+ .thenApply(tableId -> getTable(tableId));
} finally {
busyLock.leaveBusy();
}
}
- /** See {@link #createTableAsync(CreateTableParams)} for details. */
- @Deprecated(forRemoval = true)
- private CompletableFuture<Table> createTableAsyncInternal(
- int tableId,
- String name,
- String zoneName,
- Consumer<TableChange> tableInitChange
- ) {
- CompletableFuture<Table> tblFut = new CompletableFuture<>();
-
- tableAsyncInternal(name)
- .handle((tbl, tblEx) -> {
- if (tbl != null) {
- tblFut.completeExceptionally(new TableAlreadyExistsException(DEFAULT_SCHEMA_NAME, name));
- } else if (tblEx != null) {
- tblFut.completeExceptionally(tblEx);
- } else {
- if (!busyLock.enterBusy()) {
- NodeStoppingException nodeStoppingException = new NodeStoppingException();
-
- tblFut.completeExceptionally(nodeStoppingException);
-
- throw new IgniteException(nodeStoppingException);
- }
-
- try {
- distributionZoneManager.zoneIdAsyncInternal(zoneName).handle((zoneId, zoneIdEx) -> {
- if (zoneId == null) {
- tblFut.completeExceptionally(new DistributionZoneNotFoundException(zoneName));
- } else if (zoneIdEx != null) {
- tblFut.completeExceptionally(zoneIdEx);
- } else {
- if (!busyLock.enterBusy()) {
- NodeStoppingException nodeStoppingException = new NodeStoppingException();
-
- tblFut.completeExceptionally(nodeStoppingException);
-
- throw new IgniteException(nodeStoppingException);
- }
-
- try {
- cmgMgr.logicalTopology()
- .handle((cmgTopology, e) -> {
- if (e == null) {
- if (!busyLock.enterBusy()) {
- NodeStoppingException nodeStoppingException = new NodeStoppingException();
-
- tblFut.completeExceptionally(nodeStoppingException);
-
- throw new IgniteException(nodeStoppingException);
- }
-
- try {
- changeTablesConfigurationOnTableCreate(
- tableId,
- name,
- zoneId,
- tableInitChange,
- tblFut
- );
- } finally {
- busyLock.leaveBusy();
- }
- } else {
- tblFut.completeExceptionally(e);
- }
-
- return null;
- });
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- return null;
- });
- } finally {
- busyLock.leaveBusy();
- }
- }
-
- return null;
- });
-
- return tblFut;
- }
-
- /**
- * Creates a new table in {@link TablesConfiguration}.
- *
- * @param tableId Table id.
- * @param name Table name.
- * @param zoneId Distribution zone id.
- * @param tableInitChange Table changer.
- * @param tblFut Future representing pending completion of the table creation.
- */
- private void changeTablesConfigurationOnTableCreate(
- int tableId,
- String name,
- int zoneId,
- Consumer<TableChange> tableInitChange,
- CompletableFuture<Table> tblFut
- ) {
- tablesCfg.change(tablesChange -> {
- incrementTablesGeneration(tablesChange);
-
- tablesChange.changeTables(tablesListChange -> {
- if (tablesListChange.get(name) != null) {
- throw new TableAlreadyExistsException(DEFAULT_SCHEMA_NAME, name);
- }
-
- tablesListChange.create(name, (tableChange) -> {
- tableInitChange.accept(tableChange);
-
- tableChange.changeZoneId(zoneId);
-
- var extConfCh = ((ExtendedTableChange) tableChange);
-
- extConfCh.changeId(tableId);
-
- tablesChange.changeGlobalIdCounter(tableId);
-
- extConfCh.changeSchemaId(INITIAL_SCHEMA_VERSION);
-
- tableCreateFuts.put(extConfCh.id(), tblFut);
- });
- });
- }).exceptionally(t -> {
- Throwable ex = getRootCause(t);
-
- if (ex instanceof TableAlreadyExistsException) {
- tblFut.completeExceptionally(ex);
- } else {
- LOG.debug("Unable to create table [name={}]", ex, name);
-
- tblFut.completeExceptionally(ex);
-
- tableCreateFuts.values().removeIf(fut -> fut == tblFut);
- }
-
- return null;
- });
- }
-
- private static void incrementTablesGeneration(TablesChange tablesChange) {
- tablesChange.changeTablesGeneration(tablesChange.tablesGeneration() + 1);
- }
-
/**
* Alters a cluster table. If an appropriate table does not exist, a future will be completed with {@link TableNotFoundException}.
*
@@ -1701,84 +1509,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
}
- /**
- * Alter table configuration.
- *
- * @see AlterTableAddColumnParams
- * @see AlterTableDropColumnParams
- */
- @Deprecated(forRemoval = true)
- private CompletableFuture<Void> alterTableAsyncInternal(String name, Function<TableChange, Boolean> tableChange) {
- CompletableFuture<Void> tblFut = new CompletableFuture<>();
-
- tableAsync(name).thenAccept(tbl -> {
- if (tbl == null) {
- tblFut.completeExceptionally(new TableNotFoundException(DEFAULT_SCHEMA_NAME, name));
- } else {
- tablesCfg.tables().change(ch -> {
- if (ch.get(name) == null) {
- throw new TableNotFoundException(DEFAULT_SCHEMA_NAME, name);
- }
-
- ch.update(name, tblCh -> {
- if (!tableChange.apply(tblCh)) {
- return;
- }
-
- ExtendedTableChange exTblChange = (ExtendedTableChange) tblCh;
-
- exTblChange.changeSchemaId(exTblChange.schemaId() + 1);
- });
- }).whenComplete((res, t) -> {
- if (t != null) {
- Throwable ex = getRootCause(t);
-
- if (ex instanceof TableNotFoundException) {
- tblFut.completeExceptionally(ex);
- } else {
- LOG.debug("Unable to modify table [name={}]", ex, name);
-
- tblFut.completeExceptionally(ex);
- }
- } else {
- tblFut.complete(res);
- }
- });
- }
- }).exceptionally(th -> {
- tblFut.completeExceptionally(th);
-
- return null;
- });
-
- return tblFut;
- }
-
- /**
- * Gets a cause exception for a client.
- *
- * @param t Exception wrapper.
- * @return A root exception which will be acceptable to throw for public API.
- */
- //TODO: IGNITE-16051 Implement exception converter for public API.
- private @NotNull IgniteException getRootCause(Throwable t) {
- Throwable ex;
-
- if (t instanceof CompletionException) {
- if (t.getCause() instanceof ConfigurationChangeException) {
- ex = t.getCause().getCause();
- } else {
- ex = t.getCause();
- }
-
- } else {
- ex = t;
- }
-
- // TODO https://issues.apache.org/jira/browse/IGNITE-19539
- return (ex instanceof IgniteException) ? (IgniteException) ex : IgniteExceptionUtils.wrap(ex);
- }
-
/**
* Drops a table with the name specified. If appropriate table does not be found, a future will be completed with
* {@link TableNotFoundException}.
@@ -1792,55 +1522,12 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
throw new IgniteException(new NodeStoppingException());
}
try {
- return catalogManager.dropTable(params)
- .thenCompose(ignore -> dropTableAsyncInternal(params.tableName()));
+ return catalogManager.dropTable(params);
} finally {
busyLock.leaveBusy();
}
}
- /** See {@link #dropTableAsync(DropTableParams)} for details. */
- @Deprecated(forRemoval = true)
- private CompletableFuture<Void> dropTableAsyncInternal(String name) {
- return tableAsyncInternal(name).thenCompose(tbl -> {
- // In case of drop it's an optimization that allows not to fire drop-change-closure if there's no such
- // distributed table and the local config has lagged behind.
- if (tbl == null) {
- return failedFuture(new TableNotFoundException(DEFAULT_SCHEMA_NAME, name));
- }
-
- return tablesCfg
- .change(chg -> {
- incrementTablesGeneration(chg);
-
- chg
- .changeTables(tblChg -> {
- if (tblChg.get(name) == null) {
- throw new TableNotFoundException(DEFAULT_SCHEMA_NAME, name);
- }
-
- tblChg.delete(name);
- })
- .changeIndexes(idxChg -> {
- for (TableIndexView index : idxChg) {
- if (index.tableId() == tbl.tableId()) {
- idxChg.delete(index.name());
- }
- }
- });
- })
- .exceptionally(t -> {
- Throwable ex = getRootCause(t);
-
- if (!(ex instanceof TableNotFoundException)) {
- LOG.debug("Unable to drop table [name={}]", ex, name);
- }
-
- throw new CompletionException(ex);
- });
- });
- }
-
/** {@inheritDoc} */
@Override
public List<Table> tables() {
@@ -1866,7 +1553,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Future representing pending completion of the operation.
*/
private CompletableFuture<List<Table>> tablesAsyncInternal() {
- return supplyAsync(() -> inBusyLock(busyLock, this::directTableIds), ioExecutor)
+ return inBusyLock(busyLock, this::directTableIds)
.thenCompose(tableIds -> inBusyLock(busyLock, () -> {
var tableFuts = new CompletableFuture[tableIds.size()];
@@ -1897,8 +1584,18 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
*
* @return A list of direct table ids.
*/
- private List<Integer> directTableIds() {
- return configuredTablesCache.configuredTableIds();
+ private CompletableFuture<List<Integer>> directTableIds() {
+ CompletableFuture<?> res = getMetadataLocallyOnly ? completedFuture(null) : catalogManager.waitLatest();
+
+ return res.thenApplyAsync(ignore -> {
+ CatalogSchemaDescriptor schema = catalogManager.activeSchema(Long.MAX_VALUE);
+
+ if (schema == null) {
+ return List.of();
+ }
+
+ return Arrays.stream(schema.tables()).map(CatalogObjectDescriptor::id).collect(toList());
+ }, ioExecutor);
}
/**
@@ -1908,18 +1605,18 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Direct id of the table, or {@code null} if the table with the {@code tblName} has not been found.
*/
@Nullable
- private Integer directTableId(String tblName) {
- try {
- TableConfiguration exTblCfg = directProxy(tablesCfg.tables()).get(tblName);
+ private CompletableFuture<Integer> directTableId(String tblName) {
+ CompletableFuture<?> res = getMetadataLocallyOnly ? completedFuture(null) : catalogManager.waitLatest();
- if (exTblCfg == null) {
+ return res.thenApplyAsync(ignore -> {
+ CatalogTableDescriptor table = catalogManager.table(tblName, Long.MAX_VALUE);
+
+ if (table == null) {
return null;
} else {
- return exTblCfg.id().value();
+ return table.id();
}
- } catch (NoSuchElementException e) {
- return null;
- }
+ }, ioExecutor);
}
/**
@@ -1930,7 +1627,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return The future with tables map.
* @see #assignmentsUpdatedVv
*/
- private CompletableFuture<Map<Integer, TableImpl>> tablesById(long causalityToken) {
+ public CompletableFuture<Map<Integer, TableImpl>> tablesById(long causalityToken) {
return assignmentsUpdatedVv.get(causalityToken).thenCompose(v -> tablesByIdVv.get(causalityToken));
}
@@ -2055,7 +1752,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
try {
- return supplyAsync(() -> inBusyLock(busyLock, () -> directTableId(name)), ioExecutor)
+ return inBusyLock(busyLock, () -> directTableId(name))
.thenCompose(tableId -> inBusyLock(busyLock, () -> {
if (tableId == null) {
return completedFuture(null);
@@ -2137,7 +1834,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return True when the table is configured into cluster, false otherwise.
*/
private boolean isTableConfigured(int id) {
- return configuredTablesCache.isTableConfigured(id);
+ return catalogManager.table(id, catalogManager.activeCatalogVersion(Long.MAX_VALUE)) != null;
}
/**
@@ -2506,19 +2203,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
return new PartitionMover(busyLock, () -> internalTable.partitionRaftGroupService(partId));
}
- /**
- * Gets a direct accessor for the configuration distributed property. If the metadata access only locally configured the method will
- * return local property accessor.
- *
- * @param property Distributed configuration property to receive direct access.
- * @param <T> Type of the property accessor.
- * @return An accessor for distributive property.
- * @see #getMetadataLocallyOnly
- */
- private <T extends ConfigurationProperty<?>> T directProxy(T property) {
- return getMetadataLocallyOnly ? property : (T) property.directProxy();
- }
-
private static PeersAndLearners configurationFromAssignments(Collection<Assignment> assignments) {
var peers = new HashSet<String>();
var learners = new HashSet<String>();
@@ -2766,21 +2450,4 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
return zone.replicas();
}
-
- @Deprecated(forRemoval = true)
- private static CatalogDataStorageDescriptor toDataStorageDescriptor(DataStorageView config) {
- String dataRegion;
-
- try {
- Method dataRegionMethod = config.getClass().getMethod("dataRegion");
-
- dataRegionMethod.setAccessible(true);
-
- dataRegion = (String) dataRegionMethod.invoke(config);
- } catch (ReflectiveOperationException e) {
- dataRegion = e.getMessage();
- }
-
- return new CatalogDataStorageDescriptor(config.name(), dataRegion);
- }
}
[ignite-3] 04/19: Get rid of DdlCommandHandlerWrapper. Get rid of unused methods. Fix styles.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 87585a6643fb80d33216dac74a0d088e2db8a74e
Author: amashenkov <an...@gmail.com>
AuthorDate: Thu Jul 13 20:07:39 2023 +0300
Get rid of DdlCommandHandlerWrapper.
Get rid of unused methods.
Fix styles.
---
.../internal/catalog/CatalogManagerImpl.java | 2 +-
.../internal/catalog/commands/CatalogUtils.java | 3 +
.../distributionzones/DistributionZoneManager.java | 8 +-
.../apache/ignite/internal/index/IndexManager.java | 176 ++-------
.../ignite/internal/index/IndexManagerTest.java | 11 +-
.../ignite/internal/index/ItIndexManagerTest.java | 8 +-
.../testutils/SchemaToCatalogParamsConverter.java | 3 +
.../internal/sql/engine/SqlQueryProcessor.java | 4 +-
.../sql/engine/exec/ddl/DdlCommandHandler.java | 403 ++-------------------
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 114 ------
.../DdlCommandHandlerExceptionHandlingTest.java | 2 +-
.../ddl/DistributionZoneDdlCommandHandlerTest.java | 6 +-
.../internal/table/distributed/TableManager.java | 41 +--
13 files changed, 86 insertions(+), 695 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index 02e9ca4209..24f413484c 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -154,7 +154,7 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
// TODO: IGNITE-19082 Move default schema objects initialization to cluster init procedure.
CatalogSchemaDescriptor schemaPublic = new CatalogSchemaDescriptor(
- 0,
+ objectIdGen++,
DEFAULT_SCHEMA_NAME,
new CatalogTableDescriptor[0],
new CatalogIndexDescriptor[0]
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
index 7d53598e23..4243a54b3d 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/commands/CatalogUtils.java
@@ -44,18 +44,21 @@ public class CatalogUtils {
* <p>SQL`16 part 2 section 6.1 syntax rule 36
*/
public static final int DEFAULT_TIMESTAMP_PRECISION = 6;
+
/**
* Default TIME type precision: seconds.
*
* <p>SQL`16 part 2 section 6.1 syntax rule 36
*/
public static final int DEFAULT_TIME_PRECISION = 0;
+
/**
* Default DECIMAL precision is implementation-defined.
*
* <p>SQL`16 part 2 section 6.1 syntax rule 25
*/
public static final int DEFAULT_DECIMAL_PRECISION = 19;
+
/**
* Default DECIMAL scale is implementation-defined.
*
diff --git a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
index 97863daf36..8f67682499 100644
--- a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
+++ b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
@@ -98,7 +98,6 @@ import org.apache.ignite.internal.catalog.commands.AlterZoneParams;
import org.apache.ignite.internal.catalog.commands.CreateZoneParams;
import org.apache.ignite.internal.catalog.commands.DropZoneParams;
import org.apache.ignite.internal.catalog.commands.RenameZoneParams;
-import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalNode;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologyEventListener;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologyService;
@@ -409,12 +408,7 @@ public class DistributionZoneManager implements IgniteComponent {
.dataNodesAutoAdjustScaleDown(distributionZoneCfg.dataNodesAutoAdjustScaleDown())
.build())
.thenApply(ignore -> catalogManager.zone(distributionZoneCfg.name(), Long.MAX_VALUE))
- .thenCompose(zoneDescriptor -> createZone(zoneDescriptor.id(), distributionZoneCfg))
- .whenComplete((id, ex) -> {
- if (ex != null) {
- LOG.warn("Failed to create zone.", ex);
- }
- });
+ .thenCompose(zoneDescriptor -> createZone(zoneDescriptor.id(), distributionZoneCfg));
} finally {
busyLock.leaveBusy();
}
diff --git a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
index 2944ef3f6e..d14dc9dfd5 100644
--- a/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
+++ b/modules/index/src/main/java/org/apache/ignite/internal/index/IndexManager.java
@@ -33,10 +33,10 @@ import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Consumer;
import java.util.function.Function;
-import org.apache.ignite.configuration.NamedListView;
import org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.AbstractIndexCommandParams;
import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
import org.apache.ignite.internal.catalog.commands.DropIndexParams;
@@ -191,106 +191,52 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
}
/**
- * Creates index from provided configuration changer.
- *
- * @param schemaName A name of the schema to create index in.
- * @param indexName A name of the index to create.
- * @param tableName A name of the table to create index for.
- * @param failIfExists Flag indicates whether exception be thrown if index exists or not.
- * @param indexChange A consumer that suppose to change the configuration in order to provide description of an index.
- * @return A future represented the result of creation.
- */
- @Deprecated(forRemoval = true)
- public CompletableFuture<Boolean> createIndexAsync(
- String schemaName,
- String indexName,
- String tableName,
- boolean failIfExists,
- Consumer<TableIndexChange> indexChange
- ) {
- throw new UnsupportedOperationException("Method is no longer supported.");
- }
-
- /**
- * Creates sorted index from provided parameters.
+ * Creates index from provided parameters.
*/
- @Deprecated(forRemoval = true)
- public CompletableFuture<Boolean> createSortedIndexAsync(CreateSortedIndexParams params) {
+ public CompletableFuture<Boolean> createIndexAsync(AbstractIndexCommandParams params) {
if (!busyLock.enterBusy()) {
return failedFuture(new NodeStoppingException());
}
try {
- Consumer<TableIndexChange> indexChanger = tableIndexChange -> {
- tableIndexChange.convert(SortedIndexChange.class).changeColumns(colsInit -> {
- for (int i = 0; i < params.columns().size(); i++) {
- String columnName = params.columns().get(i);
- CatalogColumnCollation collation = params.collations().get(i);
- //TODO: https://issues.apache.org/jira/browse/IGNITE-17563 Pass null ordering for columns.
- colsInit.create(columnName, colInit -> colInit.changeAsc(collation.asc()));
- }
- });
- };
-
- return catalogManager.createIndex(params)
- .thenApply(ignore -> {
- CatalogIndexDescriptor index = catalogManager.index(params.indexName(), Long.MAX_VALUE);
- return index.id();
- })
- .thenCompose(indexId ->
- createIndexInternal(
- indexId,
- params.schemaName(),
- params.indexName(),
- params.tableName(),
- true,
- indexChanger
- )
- );
- } catch (Exception ex) {
- return failedFuture(ex);
+ CompletableFuture<Void> indexCreateFuture = (params instanceof CreateSortedIndexParams)
+ ? catalogManager.createIndex((CreateSortedIndexParams) params)
+ : catalogManager.createIndex((CreateHashIndexParams) params);
+
+ return indexCreateFuture.thenApply(ignore -> catalogManager.index(params.indexName(), Long.MAX_VALUE))
+ .thenCompose(index -> createIndexInternal(
+ index.id(),
+ params.schemaName(),
+ params.indexName(),
+ params.tableName(),
+ true,
+ createIndexChanger(params)
+ ));
} finally {
busyLock.leaveBusy();
}
}
- /**
- * Creates hash index from provided parameters.
- */
@Deprecated(forRemoval = true)
- public CompletableFuture<Boolean> createHashIndexAsync(CreateHashIndexParams params) {
- if (!busyLock.enterBusy()) {
- return failedFuture(new NodeStoppingException());
- }
-
- try {
- Consumer<TableIndexChange> indexChanger = tableIndexChange -> {
- tableIndexChange.convert(HashIndexChange.class)
- .changeColumnNames(params.columns().toArray(STRING_EMPTY_ARRAY));
- };
-
- return catalogManager.createIndex(params)
- .thenApply(ignore -> {
- CatalogIndexDescriptor index = catalogManager.index(params.indexName(), Long.MAX_VALUE);
- return index.id();
- })
- .thenCompose(indexId ->
- createIndexInternal(
- indexId,
- params.schemaName(),
- params.indexName(),
- params.tableName(),
- true,
- indexChanger
- )
- );
- } catch (Exception ex) {
- return failedFuture(ex);
- } finally {
- busyLock.leaveBusy();
+ private Consumer<TableIndexChange> createIndexChanger(AbstractIndexCommandParams indexParams) {
+ if (indexParams instanceof CreateSortedIndexParams) {
+ CreateSortedIndexParams params = (CreateSortedIndexParams) indexParams;
+ return tableIndexChange -> tableIndexChange.convert(SortedIndexChange.class).changeColumns(colsInit -> {
+ for (int i = 0; i < params.columns().size(); i++) {
+ String columnName = params.columns().get(i);
+ CatalogColumnCollation collation = params.collations().get(i);
+ //TODO: https://issues.apache.org/jira/browse/IGNITE-17563 Pass null ordering for columns.
+ colsInit.create(columnName, colInit -> colInit.changeAsc(collation.asc()));
+ }
+ });
+ } else {
+ CreateHashIndexParams params = (CreateHashIndexParams) indexParams;
+ return tableIndexChange -> tableIndexChange.convert(HashIndexChange.class)
+ .changeColumnNames(params.columns().toArray(STRING_EMPTY_ARRAY));
}
}
+ @Deprecated(forRemoval = true)
private CompletableFuture<Boolean> createIndexInternal(
int indexId,
String schemaName,
@@ -366,29 +312,11 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
});
return future;
- } catch (Exception ex) {
- return failedFuture(ex);
} finally {
busyLock.leaveBusy();
}
}
- /**
- * Drops the index with a given name asynchronously.
- *
- * @param schemaName A name of the schema the index belong to.
- * @param indexName A name of the index to drop.
- * @param failIfNotExists Flag, which force failure, when {@code trues} if index doen't not exists.
- * @return A future representing the result of the operation.
- */
- public CompletableFuture<Boolean> dropIndexAsync(
- String schemaName,
- String indexName,
- boolean failIfNotExists
- ) {
- throw new UnsupportedOperationException("Method is no longer supported.");
- }
-
/**
* Drops the index with a given parameters asynchronously.
*/
@@ -397,17 +325,14 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
throw new IgniteException(new NodeStoppingException());
}
try {
- CompletableFuture<Boolean> future = catalogManager.dropIndex(params)
+ return catalogManager.dropIndex(params)
.thenCompose(ignore -> dropIndexAsyncInternal(params.schemaName(), params.indexName(), true));
-
- future.whenComplete((res, ex) -> ex.printStackTrace());
-
- return future;
} finally {
busyLock.leaveBusy();
}
}
+ @Deprecated(forRemoval = true)
private CompletableFuture<Boolean> dropIndexAsyncInternal(
String schemaName,
String indexName,
@@ -456,44 +381,11 @@ public class IndexManager extends Producer<IndexEvent, IndexEventParameters> imp
});
return future;
- } catch (Exception ex) {
- return failedFuture(ex);
} finally {
busyLock.leaveBusy();
}
}
- /**
- * Gets a list of index configuration views for the specified table.
- *
- * @param tableName Table name.
- * @return List of index configuration views.
- */
- public List<TableIndexView> indexConfigurations(String tableName) {
- List<TableIndexView> res = new ArrayList<>();
- Integer targetTableId = null;
-
- NamedListView<TableView> tablesView = tablesCfg.tables().value();
-
- for (TableIndexView cfg : tablesCfg.indexes().value()) {
- if (targetTableId == null) {
- TableView tbl = findTableView(tablesView, cfg.tableId());
-
- if (tbl == null || !tableName.equals(tbl.name())) {
- continue;
- }
-
- targetTableId = cfg.tableId();
- } else if (!targetTableId.equals(cfg.tableId())) {
- continue;
- }
-
- res.add(cfg);
- }
-
- return res;
- }
-
private void validateName(String indexName) {
if (StringUtils.nullOrEmpty(indexName)) {
throw new IgniteInternalException(
diff --git a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
index 40fb2c13f7..53b2abf6d2 100644
--- a/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
+++ b/modules/index/src/test/java/org/apache/ignite/internal/index/IndexManagerTest.java
@@ -141,14 +141,15 @@ public class IndexManagerTest {
void configurationChangedWhenCreateIsInvoked() {
String indexName = "idx";
- assertThat(indexManager.createSortedIndexAsync(
+ assertThat(indexManager.createIndexAsync(
CreateSortedIndexParams.builder()
.schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
.tableName("tName")
.indexName(indexName)
.columns(List.of("c1", "c2"))
.collations(List.of(CatalogColumnCollation.ASC_NULLS_LAST, CatalogColumnCollation.DESC_NULLS_FIRST))
- .build()), willCompleteSuccessfully());
+ .build()),
+ willCompleteSuccessfully());
var expected = List.of(
Map.of(
@@ -176,7 +177,7 @@ public class IndexManagerTest {
@Test
public void createIndexWithEmptyName() {
assertThat(
- indexManager.createHashIndexAsync(
+ indexManager.createIndexAsync(
CreateHashIndexParams.builder()
.schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
.tableName("tName")
@@ -187,7 +188,7 @@ public class IndexManagerTest {
);
assertThat(
- indexManager.createSortedIndexAsync(
+ indexManager.createIndexAsync(
CreateSortedIndexParams.builder()
.schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
.tableName("tName")
@@ -229,7 +230,7 @@ public class IndexManagerTest {
return completedFuture(true);
});
- assertThat(indexManager.createSortedIndexAsync(
+ assertThat(indexManager.createIndexAsync(
CreateSortedIndexParams.builder()
.schemaName(CatalogManager.DEFAULT_SCHEMA_NAME)
.indexName(indexName)
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
index 3c7eca2188..242c4fbdca 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/index/ItIndexManagerTest.java
@@ -70,7 +70,7 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
CompletableFuture<IndexEventParameters> indexCreatedFuture = registerListener(indexManager, IndexEvent.CREATE);
- await(indexManager.createHashIndexAsync(
+ await(indexManager.createIndexAsync(
CreateHashIndexParams.builder()
.schemaName("PUBLIC")
.indexName("INAME")
@@ -93,11 +93,7 @@ public class ItIndexManagerTest extends ClusterPerClassIntegrationTest {
CompletableFuture<IndexEventParameters> indexDroppedFuture = registerListener(indexManager, IndexEvent.DROP);
- await(indexManager.dropIndexAsync(
- DropIndexParams.builder()
- .schemaName("PUBLIC")
- .indexName("INAME")
- .build()));
+ await(indexManager.dropIndexAsync(DropIndexParams.builder().schemaName("PUBLIC").indexName("INAME").build()));
{
IndexEventParameters params = await(indexDroppedFuture);
diff --git a/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java b/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java
index a00eda8c7f..8c5aff76fd 100644
--- a/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java
+++ b/modules/schema/src/testFixtures/java/org/apache/ignite/internal/schema/testutils/SchemaToCatalogParamsConverter.java
@@ -39,6 +39,7 @@ public class SchemaToCatalogParamsConverter {
* Types map.
*/
private static final EnumSet<ColumnType.ColumnTypeSpec> fixSizedTypes = EnumSet.of(
+ ColumnType.BOOLEAN.typeSpec(),
ColumnType.INT8.typeSpec(),
ColumnType.INT16.typeSpec(),
ColumnType.INT32.typeSpec(),
@@ -116,6 +117,8 @@ public class SchemaToCatalogParamsConverter {
private static org.apache.ignite.sql.ColumnType convert(ColumnType colType) {
switch (colType.typeSpec()) {
+ case BOOLEAN:
+ return org.apache.ignite.sql.ColumnType.BOOLEAN;
case INT8:
return org.apache.ignite.sql.ColumnType.INT8;
case INT16:
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
index f67294f396..a0b5c9bc24 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/SqlQueryProcessor.java
@@ -62,7 +62,7 @@ import org.apache.ignite.internal.sql.engine.exec.MailboxRegistryImpl;
import org.apache.ignite.internal.sql.engine.exec.QueryTaskExecutor;
import org.apache.ignite.internal.sql.engine.exec.QueryTaskExecutorImpl;
import org.apache.ignite.internal.sql.engine.exec.QueryValidationException;
-import org.apache.ignite.internal.sql.engine.exec.ddl.DdlCommandHandlerWrapper;
+import org.apache.ignite.internal.sql.engine.exec.ddl.DdlCommandHandler;
import org.apache.ignite.internal.sql.engine.message.MessageServiceImpl;
import org.apache.ignite.internal.sql.engine.prepare.PrepareService;
import org.apache.ignite.internal.sql.engine.prepare.PrepareServiceImpl;
@@ -249,7 +249,7 @@ public class SqlQueryProcessor implements QueryProcessor {
this.prepareSvc = prepareSvc;
- var ddlCommandHandler = new DdlCommandHandlerWrapper(
+ var ddlCommandHandler = new DdlCommandHandler(
distributionZoneManager,
tableManager,
indexManager,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index 5760dc2a85..e8ec21d5eb 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -19,97 +19,52 @@ package org.apache.ignite.internal.sql.engine.exec.ddl;
import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
-import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_ZONE_NAME;
-import static org.apache.ignite.internal.sql.engine.SqlQueryProcessor.DEFAULT_SCHEMA_NAME;
import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
+import static org.apache.ignite.lang.ErrorGroups.Sql.UNSUPPORTED_DDL_OPERATION_ERR;
-import java.util.ArrayList;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.Objects;
-import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
-import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.BiFunction;
-import java.util.function.Consumer;
-import java.util.function.Predicate;
-import java.util.stream.Collectors;
-import org.apache.ignite.configuration.NamedListView;
+import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.index.IndexManager;
-import org.apache.ignite.internal.schema.BitmaskNativeType;
-import org.apache.ignite.internal.schema.DecimalNativeType;
-import org.apache.ignite.internal.schema.NativeType;
-import org.apache.ignite.internal.schema.NativeTypeSpec;
-import org.apache.ignite.internal.schema.NumberNativeType;
-import org.apache.ignite.internal.schema.TemporalNativeType;
-import org.apache.ignite.internal.schema.VarlenNativeType;
-import org.apache.ignite.internal.schema.configuration.ColumnChange;
-import org.apache.ignite.internal.schema.configuration.ColumnTypeChange;
-import org.apache.ignite.internal.schema.configuration.ColumnView;
-import org.apache.ignite.internal.schema.configuration.PrimaryKeyView;
-import org.apache.ignite.internal.schema.configuration.TableChange;
-import org.apache.ignite.internal.schema.configuration.ValueSerializationHelper;
-import org.apache.ignite.internal.schema.configuration.defaultvalue.ConstantValueDefaultChange;
-import org.apache.ignite.internal.schema.configuration.defaultvalue.FunctionCallDefaultChange;
-import org.apache.ignite.internal.schema.configuration.defaultvalue.NullValueDefaultChange;
-import org.apache.ignite.internal.schema.configuration.index.HashIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.HashIndexView;
-import org.apache.ignite.internal.schema.configuration.index.IndexColumnView;
-import org.apache.ignite.internal.schema.configuration.index.SortedIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.SortedIndexView;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexChange;
-import org.apache.ignite.internal.schema.configuration.index.TableIndexView;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AbstractTableDdlCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterColumnCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableAddCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableDropCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterZoneRenameCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterZoneSetCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.ColumnDefinition;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateTableCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateZoneCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DdlCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DefaultValueDefinition.ConstantValue;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DefaultValueDefinition.FunctionCall;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropTableCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropZoneCommand;
-import org.apache.ignite.internal.sql.engine.schema.IgniteIndex.Collation;
-import org.apache.ignite.internal.sql.engine.type.IgniteTypeFactory;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.internal.util.ArrayUtils;
import org.apache.ignite.internal.util.StringUtils;
-import org.apache.ignite.lang.ColumnAlreadyExistsException;
-import org.apache.ignite.lang.ColumnNotFoundException;
import org.apache.ignite.lang.DistributionZoneAlreadyExistsException;
import org.apache.ignite.lang.DistributionZoneNotFoundException;
-import org.apache.ignite.lang.ErrorGroups;
-import org.apache.ignite.lang.ErrorGroups.Table;
-import org.apache.ignite.lang.IgniteStringBuilder;
-import org.apache.ignite.lang.IgniteStringFormatter;
+import org.apache.ignite.lang.IndexAlreadyExistsException;
+import org.apache.ignite.lang.IndexNotFoundException;
import org.apache.ignite.lang.TableAlreadyExistsException;
import org.apache.ignite.lang.TableNotFoundException;
-import org.apache.ignite.sql.SqlException;
/** DDL commands handler. */
public class DdlCommandHandler {
private final DistributionZoneManager distributionZoneManager;
- protected final TableManager tableManager;
+ private final TableManager tableManager;
- protected final IndexManager indexManager;
+ private final IndexManager indexManager;
private final DataStorageManager dataStorageManager;
+ private final CatalogManager catalogManager;
+
/**
* Constructor.
*/
@@ -117,12 +72,14 @@ public class DdlCommandHandler {
DistributionZoneManager distributionZoneManager,
TableManager tableManager,
IndexManager indexManager,
- DataStorageManager dataStorageManager
+ DataStorageManager dataStorageManager,
+ CatalogManager catalogManager
) {
this.distributionZoneManager = distributionZoneManager;
this.tableManager = tableManager;
this.indexManager = indexManager;
this.dataStorageManager = dataStorageManager;
+ this.catalogManager = catalogManager;
}
/** Handles ddl commands. */
@@ -138,7 +95,7 @@ public class DdlCommandHandler {
} else if (cmd instanceof AlterTableDropCommand) {
return handleAlterDropColumn((AlterTableDropCommand) cmd);
} else if (cmd instanceof AlterColumnCommand) {
- return completedFuture(true);
+ return handleAlterColumn((AlterColumnCommand) cmd);
} else if (cmd instanceof CreateIndexCommand) {
return handleCreateIndex((CreateIndexCommand) cmd);
} else if (cmd instanceof DropIndexCommand) {
@@ -258,51 +215,13 @@ public class DdlCommandHandler {
/** Handles create table command. */
private CompletableFuture<Boolean> handleCreateTable(CreateTableCommand cmd) {
- cmd.columns().stream()
- .map(ColumnDefinition::name)
- .filter(Predicate.not(new HashSet<>()::add))
- .findAny()
- .ifPresent(col -> {
- throw new SqlException(Table.TABLE_DEFINITION_ERR, "Can't create table with duplicate columns: "
- + cmd.columns().stream().map(ColumnDefinition::name).collect(Collectors.joining(", ")));
- });
-
- Consumer<TableChange> tblChanger = tableChange -> {
- tableChange.changeColumns(columnsChange -> {
- for (var col : cmd.columns()) {
- columnsChange.create(col.name(), columnChange -> convertColumnDefinition(col, columnChange));
- }
- });
-
- var colocationKeys = cmd.colocationColumns();
-
- if (nullOrEmpty(colocationKeys)) {
- colocationKeys = cmd.primaryKeyColumns();
- }
-
- var colocationKeys0 = colocationKeys;
-
- tableChange.changePrimaryKey(pkChange -> pkChange.changeColumns(cmd.primaryKeyColumns().toArray(String[]::new))
- .changeColocationColumns(colocationKeys0.toArray(String[]::new)));
- };
-
- String zoneName;
-
- if (cmd.zone() != null) {
- zoneName = cmd.zone();
- } else {
- zoneName = DEFAULT_ZONE_NAME;
- }
-
- return tableManager.createTableAsync(cmd.tableName(), zoneName, tblChanger)
- .thenApply(Objects::nonNull)
+ return tableManager.createTableAsync(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableAlreadyExistsException.class));
}
/** Handles drop table command. */
private CompletableFuture<Boolean> handleDropTable(DropTableCommand cmd) {
- return tableManager.dropTableAsync(cmd.tableName())
- .thenApply(v -> Boolean.TRUE)
+ return tableManager.dropTableAsync(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
@@ -312,7 +231,7 @@ public class DdlCommandHandler {
return completedFuture(Boolean.FALSE);
}
- return addColumnInternal(cmd.tableName(), cmd.columns())
+ return tableManager.alterTableAddColumnAsync(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
@@ -322,11 +241,17 @@ public class DdlCommandHandler {
return completedFuture(Boolean.FALSE);
}
- return dropColumnInternal(cmd.tableName(), cmd.columns())
+ return tableManager.alterTableDropColumnAsync(DdlToCatalogCommandConverter.convert(cmd))
.handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
}
- protected static BiFunction<Object, Throwable, Boolean> handleModificationResult(boolean ignoreExpectedError, Class<?> expErrCls) {
+ /** Handles alter column command. */
+ private CompletableFuture<Boolean> handleAlterColumn(AlterColumnCommand cmd) {
+ return catalogManager.alterColumn(DdlToCatalogCommandConverter.convert(cmd))
+ .handle(handleModificationResult(cmd.ifTableExists(), TableNotFoundException.class));
+ }
+
+ private static BiFunction<Object, Throwable, Boolean> handleModificationResult(boolean ignoreExpectedError, Class<?> expErrCls) {
return (val, err) -> {
if (err == null) {
return val instanceof Boolean ? (Boolean) val : Boolean.TRUE;
@@ -344,287 +269,13 @@ public class DdlCommandHandler {
/** Handles create index command. */
private CompletableFuture<Boolean> handleCreateIndex(CreateIndexCommand cmd) {
- cmd.columns().stream()
- .filter(Predicate.not(new HashSet<>()::add))
- .findAny()
- .ifPresent(col -> {
- throw new SqlException(ErrorGroups.Index.INVALID_INDEX_DEFINITION_ERR,
- "Can't create index on duplicate columns: " + String.join(", ", cmd.columns()));
- });
-
- Consumer<TableIndexChange> indexChanger = tableIndexChange -> {
- switch (cmd.type()) {
- case SORTED:
- createSortedIndexInternal(cmd, tableIndexChange.convert(SortedIndexChange.class));
-
- break;
- case HASH:
- createHashIndexInternal(cmd, tableIndexChange.convert(HashIndexChange.class));
-
- break;
- default:
- throw new AssertionError("Unknown index type [type=" + cmd.type() + "]");
- }
- };
-
- return indexManager.createIndexAsync(
- cmd.schemaName(),
- cmd.indexName(),
- cmd.tableName(),
- !cmd.ifNotExists(),
- indexChanger);
+ return indexManager.createIndexAsync(DdlToCatalogCommandConverter.convert(cmd))
+ .handle(handleModificationResult(cmd.ifNotExists(), IndexAlreadyExistsException.class));
}
/** Handles drop index command. */
private CompletableFuture<Boolean> handleDropIndex(DropIndexCommand cmd) {
- return indexManager.dropIndexAsync(cmd.schemaName(), cmd.indexName(), !cmd.ifNotExists());
- }
-
- /**
- * Creates sorted index.
- *
- * @param cmd Create index command.
- * @param indexChange Index configuration changer.
- */
- private void createSortedIndexInternal(CreateIndexCommand cmd, SortedIndexChange indexChange) {
- indexChange.changeColumns(colsInit -> {
- for (int i = 0; i < cmd.columns().size(); i++) {
- String columnName = cmd.columns().get(i);
- Collation collation = cmd.collations().get(i);
- //TODO: https://issues.apache.org/jira/browse/IGNITE-17563 Pass null ordering for columns.
- colsInit.create(columnName, colInit -> colInit.changeAsc(collation.asc));
- }
- });
- }
-
- /**
- * Creates hash index.
- *
- * @param cmd Create index command.
- * @param indexChange Index configuration changer.
- */
- private void createHashIndexInternal(CreateIndexCommand cmd, HashIndexChange indexChange) {
- indexChange.changeColumnNames(cmd.columns().toArray(ArrayUtils.STRING_EMPTY_ARRAY));
- }
-
- /**
- * Adds a column according to the column definition.
- *
- * @param fullName Table with schema name.
- * @param colsDef Columns defenitions.
- * @return {@code true} if the full columns set is applied successfully. Otherwise, returns {@code false}.
- */
- private CompletableFuture<Boolean> addColumnInternal(String fullName, List<ColumnDefinition> colsDef) {
- AtomicBoolean retUsr = new AtomicBoolean(true);
-
- return tableManager.alterTableAsync(
- fullName,
- chng -> {
- AtomicBoolean retTbl = new AtomicBoolean();
-
- chng.changeColumns(cols -> {
- retUsr.set(true); // Reset state if closure have been restarted.
-
- Set<String> colNamesToOrders = columnNames(chng.columns());
-
- colsDef.stream()
- .filter(k -> colNamesToOrders.contains(k.name()))
- .findAny()
- .ifPresent(c -> {
- throw new ColumnAlreadyExistsException(c.name());
- });
-
- for (ColumnDefinition col : colsDef) {
- cols.create(col.name(), colChg -> convertColumnDefinition(col, colChg));
- }
-
- retTbl.set(!colsDef.isEmpty());
- });
-
- return retTbl.get();
- }
- ).thenApply(v -> retUsr.get());
- }
-
- private void convertColumnDefinition(ColumnDefinition definition, ColumnChange columnChange) {
- NativeType columnType = IgniteTypeFactory.relDataTypeToNative(definition.type());
-
- columnChange.changeType(columnTypeChange -> convert(columnType, columnTypeChange));
- columnChange.changeNullable(definition.nullable());
- columnChange.changeDefaultValueProvider(defaultChange -> {
- switch (definition.defaultValueDefinition().type()) {
- case CONSTANT:
- ConstantValue constantValue = definition.defaultValueDefinition();
-
- var val = constantValue.value();
-
- if (val != null) {
- defaultChange.convert(ConstantValueDefaultChange.class)
- .changeDefaultValue(ValueSerializationHelper.toString(val, columnType));
- } else {
- defaultChange.convert(NullValueDefaultChange.class);
- }
-
- break;
- case FUNCTION_CALL:
- FunctionCall functionCall = definition.defaultValueDefinition();
-
- defaultChange.convert(FunctionCallDefaultChange.class)
- .changeFunctionName(functionCall.functionName());
-
- break;
- default:
- throw new IllegalStateException("Unknown default value definition type [type="
- + definition.defaultValueDefinition().type() + ']');
- }
- });
- }
-
- /**
- * Drops a column(s) exceptional behavior depends on {@code colExist} flag.
- *
- * @param tableName Table name.
- * @param colNames Columns definitions.
- * @return {@code true} if the full columns set is applied successfully. Otherwise, returns {@code false}.
- */
- private CompletableFuture<Boolean> dropColumnInternal(String tableName, Set<String> colNames) {
- AtomicBoolean ret = new AtomicBoolean(true);
-
- return tableManager.alterTableAsync(
- tableName,
- chng -> {
- chng.changeColumns(cols -> {
- ret.set(true); // Reset state if closure have been restarted.
-
- PrimaryKeyView priKey = chng.primaryKey();
-
- Set<String> colNamesToOrders = columnNames(chng.columns());
-
- Set<String> colNames0 = new HashSet<>();
-
- Set<String> primaryCols = Set.of(priKey.columns());
-
- reportIndexedColumns(tableName, colNames, primaryCols);
-
- for (String colName : colNames) {
- if (!colNamesToOrders.contains(colName)) {
- ret.set(false);
-
- throw new ColumnNotFoundException(DEFAULT_SCHEMA_NAME, tableName, colName);
- } else {
- colNames0.add(colName);
- }
-
- if (primaryCols.contains(colName)) {
- throw new SqlException(STMT_VALIDATION_ERR, IgniteStringFormatter
- .format("Can`t delete column, belongs to primary key: [name={}]", colName));
- }
- }
-
- colNames0.forEach(cols::delete);
- });
-
- return ret.get();
- }).thenApply(v -> ret.get());
- }
-
- private void reportIndexedColumns(String tableName, Set<String> colNames, Set<String> pkColNames) throws SqlException {
- Map<String, List<String>> indexedColumns = new HashMap<>();
-
- for (TableIndexView idxCfg : indexManager.indexConfigurations(tableName)) {
- if (idxCfg instanceof SortedIndexView) {
- for (IndexColumnView colView : ((SortedIndexView) idxCfg).columns()) {
- if (colNames.contains(colView.name()) && !pkColNames.contains(colView.name())) {
- indexedColumns.computeIfAbsent(colView.name(), v -> new ArrayList<>()).add(idxCfg.name());
- }
- }
- } else if (idxCfg instanceof HashIndexView) {
- for (String colName : ((HashIndexView) idxCfg).columnNames()) {
- if (colNames.contains(colName) && !pkColNames.contains(colName)) {
- indexedColumns.computeIfAbsent(colName, v -> new ArrayList<>()).add(idxCfg.name());
- }
- }
- }
- }
-
- if (indexedColumns.isEmpty()) {
- return;
- }
-
- IgniteStringBuilder sb = new IgniteStringBuilder("Can`t delete column(s). ");
-
- for (Entry<String, List<String>> e : indexedColumns.entrySet()) {
- sb.app("Column ").app(e.getKey()).app(" is used by indexes ").app(e.getValue()).app(". ");
- }
-
- throw new SqlException(STMT_VALIDATION_ERR, sb.toString());
- }
-
- private static void convert(NativeType colType, ColumnTypeChange colTypeChg) {
- NativeTypeSpec spec = colType.spec();
- String typeName = spec.name().toUpperCase();
-
- colTypeChg.changeType(typeName);
-
- switch (spec) {
- case BOOLEAN:
- case INT8:
- case INT16:
- case INT32:
- case INT64:
- case FLOAT:
- case DOUBLE:
- case DATE:
- case UUID:
- // do nothing
- break;
-
- case BITMASK:
- BitmaskNativeType bitmaskColType = (BitmaskNativeType) colType;
-
- colTypeChg.changeLength(bitmaskColType.bits());
-
- break;
-
- case BYTES:
- case STRING:
- VarlenNativeType varLenColType = (VarlenNativeType) colType;
-
- colTypeChg.changeLength(varLenColType.length());
-
- break;
-
- case DECIMAL:
- DecimalNativeType numColType = (DecimalNativeType) colType;
-
- colTypeChg.changePrecision(numColType.precision());
- colTypeChg.changeScale(numColType.scale());
-
- break;
-
- case NUMBER:
- NumberNativeType numType = (NumberNativeType) colType;
-
- colTypeChg.changePrecision(numType.precision());
-
- break;
-
- case TIME:
- case DATETIME:
- case TIMESTAMP:
- TemporalNativeType temporalColType = (TemporalNativeType) colType;
-
- colTypeChg.changePrecision(temporalColType.precision());
-
- break;
-
- default:
- throw new IllegalArgumentException("Unknown type " + colType.spec().name());
- }
- }
-
- /** Column names set. */
- private static Set<String> columnNames(NamedListView<? extends ColumnView> cols) {
- return new HashSet<>(cols.namedListKeys());
+ return indexManager.dropIndexAsync(DdlToCatalogCommandConverter.convert(cmd))
+ .handle(handleModificationResult(cmd.ifNotExists(), IndexNotFoundException.class));
}
}
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
deleted file mode 100644
index 7fbd884f9b..0000000000
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
+++ /dev/null
@@ -1,114 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.internal.sql.engine.exec.ddl;
-
-import static java.util.concurrent.CompletableFuture.failedFuture;
-import static org.apache.ignite.lang.ErrorGroups.Sql.UNSUPPORTED_DDL_OPERATION_ERR;
-
-import java.util.Objects;
-import java.util.concurrent.CompletableFuture;
-import org.apache.ignite.internal.catalog.CatalogManager;
-import org.apache.ignite.internal.catalog.commands.AbstractIndexCommandParams;
-import org.apache.ignite.internal.catalog.commands.CreateHashIndexParams;
-import org.apache.ignite.internal.catalog.commands.CreateSortedIndexParams;
-import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
-import org.apache.ignite.internal.index.IndexManager;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterColumnCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableAddCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableDropCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateIndexCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateTableCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DdlCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DropIndexCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DropTableCommand;
-import org.apache.ignite.internal.storage.DataStorageManager;
-import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.lang.IgniteInternalCheckedException;
-import org.apache.ignite.lang.IndexAlreadyExistsException;
-import org.apache.ignite.lang.IndexNotFoundException;
-import org.apache.ignite.lang.TableAlreadyExistsException;
-import org.apache.ignite.lang.TableNotFoundException;
-
-/**
- * Wrapper for DDL command handler passes DDL commands to CatalogService.
- * TODO: IGNITE-19082 Drop this wrapper when all the versioned schema stuff will be moved from Configuration to Catalog.
- */
-public class DdlCommandHandlerWrapper extends DdlCommandHandler {
-
- private final CatalogManager catalogManager;
-
- /**
- * Constructor.
- */
- public DdlCommandHandlerWrapper(
- DistributionZoneManager distributionZoneManager,
- TableManager tableManager,
- IndexManager indexManager,
- DataStorageManager dataStorageManager,
- CatalogManager catalogManager
- ) {
- super(distributionZoneManager, tableManager, indexManager, dataStorageManager);
-
- this.catalogManager = Objects.requireNonNull(catalogManager, "Catalog service");
- }
-
- /** Handles ddl commands. */
- @Override
- public CompletableFuture<Boolean> handle(DdlCommand cmd) {
- // TODO IGNITE-19082 replace TableManager with CatalogManager calls.
- if (cmd instanceof CreateTableCommand) {
- return tableManager.createTableAsync(DdlToCatalogCommandConverter.convert((CreateTableCommand) cmd))
- .handle(handleModificationResult(((CreateTableCommand) cmd).ifTableExists(), TableAlreadyExistsException.class));
- } else if (cmd instanceof DropTableCommand) {
- return tableManager.dropTableAsync(DdlToCatalogCommandConverter.convert((DropTableCommand) cmd))
- .handle(handleModificationResult(((DropTableCommand) cmd).ifTableExists(), TableNotFoundException.class));
- } else if (cmd instanceof AlterTableAddCommand) {
- AlterTableAddCommand addCommand = (AlterTableAddCommand) cmd;
-
- return tableManager.alterTableAddColumnAsync(DdlToCatalogCommandConverter.convert(addCommand))
- .handle(handleModificationResult(addCommand.ifTableExists(), TableNotFoundException.class));
- } else if (cmd instanceof AlterTableDropCommand) {
- AlterTableDropCommand dropCommand = (AlterTableDropCommand) cmd;
-
- return tableManager.alterTableDropColumnAsync(DdlToCatalogCommandConverter.convert(dropCommand))
- .handle(handleModificationResult(dropCommand.ifTableExists(), TableNotFoundException.class));
- } else if (cmd instanceof AlterColumnCommand) {
- return catalogManager.alterColumn(DdlToCatalogCommandConverter.convert((AlterColumnCommand) cmd))
- .handle(handleModificationResult(((AlterColumnCommand) cmd).ifTableExists(), TableNotFoundException.class));
- } else if (cmd instanceof CreateIndexCommand) {
- AbstractIndexCommandParams params = DdlToCatalogCommandConverter.convert((CreateIndexCommand) cmd);
-
- if (params instanceof CreateSortedIndexParams) {
- return indexManager.createSortedIndexAsync((CreateSortedIndexParams) params)
- .handle(handleModificationResult(((CreateIndexCommand) cmd).ifNotExists(), IndexAlreadyExistsException.class));
- } else if (params instanceof CreateHashIndexParams) {
- return indexManager.createHashIndexAsync((CreateHashIndexParams) params)
- .handle(handleModificationResult(((CreateIndexCommand) cmd).ifNotExists(), IndexAlreadyExistsException.class));
- }
-
- return failedFuture(new IgniteInternalCheckedException(UNSUPPORTED_DDL_OPERATION_ERR, "Unsupported DDL operation ["
- + "cmdName=" + cmd.getClass().getSimpleName() + "]"));
- } else if (cmd instanceof DropIndexCommand) {
- return indexManager.dropIndexAsync(DdlToCatalogCommandConverter.convert((DropIndexCommand) cmd))
- .handle(handleModificationResult(((DropIndexCommand) cmd).ifNotExists(), IndexNotFoundException.class));
- }
-
- // Handle other commands in usual way.
- return super.handle(cmd);
- }
-}
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
index b9e810ecd0..26bf4c8f28 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
@@ -125,7 +125,7 @@ public class DdlCommandHandlerExceptionHandlingTest extends IgniteAbstractTest {
"node"
);
- commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager);
+ commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager, catalogManager);
}
@AfterEach
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
index 8b52467311..d0a057b241 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DistributionZoneDdlCommandHandlerTest.java
@@ -26,6 +26,7 @@ import static org.junit.jupiter.api.Assertions.assertThrows;
import java.util.Arrays;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicReference;
+import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.index.IndexManager;
@@ -65,6 +66,9 @@ public class DistributionZoneDdlCommandHandlerTest extends IgniteAbstractTest {
@Mock
private DataStorageManager dataStorageManager;
+ @Mock
+ private CatalogManager catalogManager;
+
/** DDL commands handler. */
private DdlCommandHandler commandHandler;
@@ -79,7 +83,7 @@ public class DistributionZoneDdlCommandHandlerTest extends IgniteAbstractTest {
return CompletableFuture.completedFuture(null);
});
- commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager);
+ commandHandler = new DdlCommandHandler(distributionZoneManager, tableManager, indexManager, dataStorageManager, catalogManager);
}
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index 6a7b8b7721..e2e5ce1092 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -1478,25 +1478,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
);
}
- /**
- * Creates a new table with the given {@code name} asynchronously. If a table with the same name already exists, a future will be
- * completed with {@link TableAlreadyExistsException}.
- *
- * @param name Table name.
- * @param zoneName Distribution zone name.
- * @param tableInitChange Table changer.
- * @return Future representing pending completion of the operation.
- * @throws IgniteException If an unspecified platform exception has happened internally. Is thrown when:
- * <ul>
- * <li>the node is stopping.</li>
- * </ul>
- * @see TableAlreadyExistsException
- */
- @Deprecated
- public CompletableFuture<Table> createTableAsync(String name, String zoneName, Consumer<TableChange> tableInitChange) {
- throw new UnsupportedOperationException("Method is no longer supported.");
- }
-
/**
* Creates a new table from parameters.
*
@@ -1504,7 +1485,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Future representing pending completion of the operation.
* @see TableAlreadyExistsException
*/
- @Deprecated(forRemoval = true)
public CompletableFuture<Table> createTableAsync(CreateTableParams parameters) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
@@ -1713,7 +1693,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Future representing pending completion of the operation.
* @see TableNotFoundException
*/
- @Deprecated
public CompletableFuture<Void> alterTableAddColumnAsync(AlterTableAddColumnParams params) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
@@ -1733,7 +1712,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Future representing pending completion of the operation.
* @see TableNotFoundException
*/
- @Deprecated
public CompletableFuture<Void> alterTableDropColumnAsync(AlterTableDropColumnParams params) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
@@ -1824,23 +1802,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
return (ex instanceof IgniteException) ? (IgniteException) ex : IgniteExceptionUtils.wrap(ex);
}
- /**
- * Drops a table with the name specified. If appropriate table does not be found, a future will be completed with
- * {@link TableNotFoundException}.
- *
- * @param name Table name.
- * @return Future representing pending completion of the operation.
- * @throws IgniteException If an unspecified platform exception has happened internally. Is thrown when:
- * <ul>
- * <li>the node is stopping.</li>
- * </ul>
- * @see TableNotFoundException
- */
- @Deprecated(forRemoval = true)
- public CompletableFuture<Void> dropTableAsync(String name) {
- throw new UnsupportedOperationException("Method is no longer supported.");
- }
-
/**
* Drops a table with the name specified. If appropriate table does not be found, a future will be completed with
* {@link TableNotFoundException}.
@@ -1849,7 +1810,6 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
* @return Future representing pending completion of the operation.
* @see TableNotFoundException
*/
- @Deprecated
public CompletableFuture<Void> dropTableAsync(DropTableParams params) {
if (!busyLock.enterBusy()) {
throw new IgniteException(new NodeStoppingException());
@@ -1863,6 +1823,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
/** See {@link #dropTableAsync(DropTableParams)} for details. */
+ @Deprecated(forRemoval = true)
private CompletableFuture<Void> dropTableAsyncInternal(String name) {
return tableAsyncInternal(name).thenCompose(tbl -> {
// In case of drop it's an optimization that allows not to fire drop-change-closure if there's no such
[ignite-3] 06/19: Resolve conflicts.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 9a9c51b61a8da694e06437e7cccfd388f2f0a968
Author: amashenkov <an...@gmail.com>
AuthorDate: Wed Jul 19 15:12:15 2023 +0300
Resolve conflicts.
---
.../org/apache/ignite/internal/schema/CatalogDescriptorUtils.java | 1 +
.../apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java | 2 +-
.../org/apache/ignite/internal/table/distributed/TableManager.java | 4 ++--
3 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
index 54dca3f1b1..bcff576238 100644
--- a/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
+++ b/modules/schema/src/main/java/org/apache/ignite/internal/schema/CatalogDescriptorUtils.java
@@ -254,6 +254,7 @@ public class CatalogDescriptorUtils {
colTypeChg.changeType(typeName);
switch (spec) {
+ case BOOLEAN:
case INT8:
case INT16:
case INT32:
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
index e8ec21d5eb..9c9b9622bc 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandler.java
@@ -21,7 +21,6 @@ import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import static org.apache.ignite.internal.util.CollectionUtils.nullOrEmpty;
import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
-import static org.apache.ignite.lang.ErrorGroups.Sql.UNSUPPORTED_DDL_OPERATION_ERR;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionException;
@@ -52,6 +51,7 @@ import org.apache.ignite.lang.IndexAlreadyExistsException;
import org.apache.ignite.lang.IndexNotFoundException;
import org.apache.ignite.lang.TableAlreadyExistsException;
import org.apache.ignite.lang.TableNotFoundException;
+import org.apache.ignite.sql.SqlException;
/** DDL commands handler. */
public class DdlCommandHandler {
diff --git a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
index e2e5ce1092..41f3a4d886 100644
--- a/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
+++ b/modules/table/src/main/java/org/apache/ignite/internal/table/distributed/TableManager.java
@@ -46,7 +46,7 @@ import static org.apache.ignite.internal.utils.RebalanceUtil.extractPartitionNum
import static org.apache.ignite.internal.utils.RebalanceUtil.extractTableId;
import static org.apache.ignite.internal.utils.RebalanceUtil.pendingPartAssignmentsKey;
import static org.apache.ignite.internal.utils.RebalanceUtil.stablePartAssignmentsKey;
-import static org.apache.ignite.lang.ErrorGroups.Sql.DROP_IDX_COLUMN_CONSTRAINT_ERR;
+import static org.apache.ignite.lang.ErrorGroups.Sql.STMT_VALIDATION_ERR;
import it.unimi.dsi.fastutil.ints.Int2ObjectOpenHashMap;
import java.io.IOException;
@@ -2858,7 +2858,7 @@ public class TableManager extends Producer<TableEvent, TableEventParameters> imp
}
if (primaryCols.contains(colName)) {
- throw new SqlException(DROP_IDX_COLUMN_CONSTRAINT_ERR, IgniteStringFormatter
+ throw new SqlException(STMT_VALIDATION_ERR, IgniteStringFormatter
.format("Can`t delete column, belongs to primary key: [name={}]", colName));
}
}
[ignite-3] 01/19: Mirror DistributionZone changes from Config to Catalog.
Posted by am...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
amashenkov pushed a commit to branch ignite-19499
in repository https://gitbox.apache.org/repos/asf/ignite-3.git
commit 5cf3c4161eb33a828d570d436434d8091db359d4
Author: amashenkov <an...@gmail.com>
AuthorDate: Tue Jul 4 23:29:33 2023 +0300
Mirror DistributionZone changes from Config to Catalog.
---
.../internal/catalog/CatalogManagerImpl.java | 6 +-
.../catalog/descriptors/CatalogZoneDescriptor.java | 3 +
modules/distribution-zones/build.gradle | 2 +
.../distributionzones/DistributionZoneManager.java | 74 ++++++++++++++++++++--
.../BaseDistributionZoneManagerTest.java | 1 +
...ibutionZoneManagerConfigurationChangesTest.java | 1 +
.../DistributionZoneManagerTest.java | 1 +
.../DistributionZoneMockTest.java | 2 +
.../DistributionZonesTestUtil.java | 34 ++++++++++
.../storage/ItRebalanceDistributedTest.java | 1 +
...niteDistributionZoneManagerNodeRestartTest.java | 2 +
.../runner/app/ItIgniteNodeRestartTest.java | 15 +++--
.../org/apache/ignite/internal/app/IgniteImpl.java | 21 +++---
.../engine/exec/ddl/DdlCommandHandlerWrapper.java | 34 ----------
.../DdlCommandHandlerExceptionHandlingTest.java | 28 ++++----
15 files changed, 151 insertions(+), 74 deletions(-)
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
index a6b29af3cb..02e9ca4209 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java
@@ -150,18 +150,18 @@ public class CatalogManagerImpl extends Producer<CatalogEvent, CatalogEventParam
@Override
public void start() {
- int objectIdGen = 0;
+ int objectIdGen = 1;
// TODO: IGNITE-19082 Move default schema objects initialization to cluster init procedure.
CatalogSchemaDescriptor schemaPublic = new CatalogSchemaDescriptor(
- objectIdGen++,
+ 0,
DEFAULT_SCHEMA_NAME,
new CatalogTableDescriptor[0],
new CatalogIndexDescriptor[0]
);
CatalogZoneDescriptor defaultZone = new CatalogZoneDescriptor(
- objectIdGen++,
+ 0,
DEFAULT_ZONE_NAME,
25,
1,
diff --git a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogZoneDescriptor.java b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogZoneDescriptor.java
index fd636e2b68..01fd779cb0 100644
--- a/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogZoneDescriptor.java
+++ b/modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogZoneDescriptor.java
@@ -76,6 +76,9 @@ public class CatalogZoneDescriptor extends CatalogObjectDescriptor {
this.dataNodesAutoAdjustScaleUp = dataNodesAutoAdjustScaleUp;
this.dataNodesAutoAdjustScaleDown = dataNodesAutoAdjustScaleDown;
this.filter = filter;
+
+ // TODO: IGNITE-19719 Fix it
+ this.dataStorage = new CatalogDataStorageDescriptor("aipersist", "default");
}
/**
diff --git a/modules/distribution-zones/build.gradle b/modules/distribution-zones/build.gradle
index 54c78c763f..ec90a2420d 100644
--- a/modules/distribution-zones/build.gradle
+++ b/modules/distribution-zones/build.gradle
@@ -26,6 +26,7 @@ dependencies {
annotationProcessor project(":ignite-configuration-annotation-processor")
annotationProcessor libs.auto.service
+ implementation project(':ignite-catalog')
implementation project(':ignite-core')
implementation project(':ignite-configuration-api')
implementation project(':ignite-api')
@@ -66,6 +67,7 @@ dependencies {
testFixturesImplementation libs.mockito.core
testFixturesImplementation libs.mockito.junit
testFixturesImplementation libs.hamcrest.core
+ testFixturesImplementation project(':ignite-catalog')
testFixturesImplementation project(':ignite-raft-api')
testFixturesImplementation project(':ignite-metastorage')
testFixturesImplementation project(':ignite-schema')
diff --git a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
index d61f044560..97863daf36 100644
--- a/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
+++ b/modules/distribution-zones/src/main/java/org/apache/ignite/internal/distributionzones/DistributionZoneManager.java
@@ -18,6 +18,7 @@
package org.apache.ignite.internal.distributionzones;
import static java.util.Collections.emptySet;
+import static java.util.concurrent.CompletableFuture.allOf;
import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.failedFuture;
import static java.util.concurrent.CompletableFuture.supplyAsync;
@@ -92,6 +93,12 @@ import org.apache.ignite.configuration.notifications.ConfigurationListener;
import org.apache.ignite.configuration.notifications.ConfigurationNamedListListener;
import org.apache.ignite.configuration.notifications.ConfigurationNotificationEvent;
import org.apache.ignite.configuration.validation.ConfigurationValidationException;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.commands.AlterZoneParams;
+import org.apache.ignite.internal.catalog.commands.CreateZoneParams;
+import org.apache.ignite.internal.catalog.commands.DropZoneParams;
+import org.apache.ignite.internal.catalog.commands.RenameZoneParams;
+import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalNode;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologyEventListener;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologyService;
@@ -185,6 +192,9 @@ public class DistributionZoneManager implements IgniteComponent {
/** Tables configuration. */
private final TablesConfiguration tablesConfiguration;
+ /** Catalog service. */
+ private final CatalogManager catalogManager;
+
/** Meta Storage manager. */
private final MetaStorageManager metaStorageManager;
@@ -256,6 +266,7 @@ public class DistributionZoneManager implements IgniteComponent {
*
* @param zonesConfiguration Distribution zones configuration.
* @param tablesConfiguration Tables configuration.
+ * @param catalogManager Catalog manager.
* @param metaStorageManager Meta Storage manager.
* @param logicalTopologyService Logical topology service.
* @param vaultMgr Vault manager.
@@ -264,6 +275,7 @@ public class DistributionZoneManager implements IgniteComponent {
public DistributionZoneManager(
DistributionZonesConfiguration zonesConfiguration,
TablesConfiguration tablesConfiguration,
+ CatalogManager catalogManager,
MetaStorageManager metaStorageManager,
LogicalTopologyService logicalTopologyService,
VaultManager vaultMgr,
@@ -274,6 +286,7 @@ public class DistributionZoneManager implements IgniteComponent {
this.metaStorageManager = metaStorageManager;
this.logicalTopologyService = logicalTopologyService;
this.vaultMgr = vaultMgr;
+ this.catalogManager = catalogManager;
this.topologyWatchListener = createMetastorageTopologyListener();
@@ -386,9 +399,37 @@ public class DistributionZoneManager implements IgniteComponent {
}
try {
- CompletableFuture<Integer> fut = new CompletableFuture<>();
+ return catalogManager.createDistributionZone(CreateZoneParams.builder()
+ .zoneName(distributionZoneCfg.name())
+ .partitions(distributionZoneCfg.partitions())
+ .filter(distributionZoneCfg.filter())
+ .replicas(distributionZoneCfg.replicas())
+ .dataNodesAutoAdjust(distributionZoneCfg.dataNodesAutoAdjust())
+ .dataNodesAutoAdjustScaleUp(distributionZoneCfg.dataNodesAutoAdjustScaleUp())
+ .dataNodesAutoAdjustScaleDown(distributionZoneCfg.dataNodesAutoAdjustScaleDown())
+ .build())
+ .thenApply(ignore -> catalogManager.zone(distributionZoneCfg.name(), Long.MAX_VALUE))
+ .thenCompose(zoneDescriptor -> createZone(zoneDescriptor.id(), distributionZoneCfg))
+ .whenComplete((id, ex) -> {
+ if (ex != null) {
+ LOG.warn("Failed to create zone.", ex);
+ }
+ });
+ } finally {
+ busyLock.leaveBusy();
+ }
+ }
- int[] zoneIdContainer = new int[1];
+ private CompletableFuture<Integer> createZone(
+ int intZoneId,
+ DistributionZoneConfigurationParameters distributionZoneCfg
+ ) {
+ if (!busyLock.enterBusy()) {
+ return failedFuture(new NodeStoppingException());
+ }
+
+ try {
+ CompletableFuture<Integer> fut = new CompletableFuture<>();
zonesConfiguration.change(zonesChange -> zonesChange.changeDistributionZones(zonesListChange -> {
try {
@@ -427,11 +468,9 @@ public class DistributionZoneManager implements IgniteComponent {
zoneChange.changeDataNodesAutoAdjustScaleDown(distributionZoneCfg.dataNodesAutoAdjustScaleDown());
}
- int intZoneId = zonesChange.globalIdCounter() + 1;
zonesChange.changeGlobalIdCounter(intZoneId);
zoneChange.changeZoneId(intZoneId);
- zoneIdContainer[0] = intZoneId;
});
} catch (ConfigurationNodeAlreadyExistException e) {
throw new DistributionZoneAlreadyExistsException(distributionZoneCfg.name(), e);
@@ -445,7 +484,7 @@ public class DistributionZoneManager implements IgniteComponent {
ConfigurationValidationException.class)
);
} else {
- fut.complete(zoneIdContainer[0]);
+ fut.complete(intZoneId);
}
});
@@ -496,6 +535,23 @@ public class DistributionZoneManager implements IgniteComponent {
try {
CompletableFuture<Void> fut = new CompletableFuture<>();
+ CompletableFuture<Void> catalogFut = catalogManager.alterDistributionZone(AlterZoneParams.builder()
+ .zoneName(name)
+ .partitions(distributionZoneCfg.partitions())
+ .replicas(distributionZoneCfg.replicas())
+ .filter(distributionZoneCfg.filter())
+ .dataNodesAutoAdjust(distributionZoneCfg.dataNodesAutoAdjust())
+ .dataNodesAutoAdjustScaleUp(distributionZoneCfg.dataNodesAutoAdjustScaleUp())
+ .dataNodesAutoAdjustScaleDown(distributionZoneCfg.dataNodesAutoAdjustScaleDown())
+ .build());
+
+ if (!name.equals(distributionZoneCfg.name())) {
+ catalogFut = catalogFut.thenCompose(ignore -> catalogManager.renameDistributionZone(RenameZoneParams.builder()
+ .zoneName(name)
+ .newZoneName(distributionZoneCfg.name())
+ .build()));
+ }
+
CompletableFuture<Void> change;
if (DEFAULT_ZONE_NAME.equals(name)) {
@@ -524,7 +580,7 @@ public class DistributionZoneManager implements IgniteComponent {
}));
}
- change.whenComplete((res, e) -> {
+ allOf(catalogFut, change).whenComplete((res, e) -> {
if (e != null) {
fut.completeExceptionally(
unwrapDistributionZoneException(
@@ -603,7 +659,11 @@ public class DistributionZoneManager implements IgniteComponent {
}
});
- return fut;
+ CompletableFuture<Void> dropZoneFut = catalogManager.dropDistributionZone(DropZoneParams.builder()
+ .zoneName(name)
+ .build());
+
+ return allOf(dropZoneFut, fut);
} finally {
busyLock.leaveBusy();
}
diff --git a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/BaseDistributionZoneManagerTest.java b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/BaseDistributionZoneManagerTest.java
index 9c69f09930..a9177185e0 100644
--- a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/BaseDistributionZoneManagerTest.java
+++ b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/BaseDistributionZoneManagerTest.java
@@ -126,6 +126,7 @@ public class BaseDistributionZoneManagerTest extends BaseIgniteAbstractTest {
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfiguration,
+ DistributionZonesTestUtil.mockCatalog(),
metaStorageManager,
new LogicalTopologyServiceImpl(topology, cmgManager),
vaultMgr,
diff --git a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerConfigurationChangesTest.java b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerConfigurationChangesTest.java
index 713ee78a4e..8f58ea7b78 100644
--- a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerConfigurationChangesTest.java
+++ b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerConfigurationChangesTest.java
@@ -146,6 +146,7 @@ public class DistributionZoneManagerConfigurationChangesTest extends IgniteAbstr
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfiguration,
+ DistributionZonesTestUtil.mockCatalog(),
metaStorageManager,
logicalTopologyService,
vaultMgr,
diff --git a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerTest.java b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerTest.java
index 5bd1324a97..8c1bb0fc2a 100644
--- a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerTest.java
+++ b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneManagerTest.java
@@ -99,6 +99,7 @@ class DistributionZoneManagerTest extends IgniteAbstractTest {
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfiguration,
+ DistributionZonesTestUtil.mockCatalog(),
null,
null,
null,
diff --git a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneMockTest.java b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneMockTest.java
index 75db4d543f..17ede4c011 100644
--- a/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneMockTest.java
+++ b/modules/distribution-zones/src/test/java/org/apache/ignite/internal/distributionzones/DistributionZoneMockTest.java
@@ -21,6 +21,7 @@ import static org.junit.jupiter.api.Assertions.assertNull;
import static org.mockito.Mockito.mock;
import java.util.concurrent.TimeUnit;
+import org.apache.ignite.internal.catalog.CatalogManager;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalTopologyService;
import org.apache.ignite.internal.configuration.testframework.ConfigurationExtension;
import org.apache.ignite.internal.configuration.testframework.InjectConfiguration;
@@ -44,6 +45,7 @@ public class DistributionZoneMockTest {
DistributionZoneManager zoneMgr = new DistributionZoneManager(
zonesConfiguration,
mock(TablesConfiguration.class),
+ mock(CatalogManager.class),
mock(MetaStorageManager.class),
mock(LogicalTopologyService.class),
mock(VaultManager.class),
diff --git a/modules/distribution-zones/src/testFixtures/java/org/apache/ignite/internal/distributionzones/DistributionZonesTestUtil.java b/modules/distribution-zones/src/testFixtures/java/org/apache/ignite/internal/distributionzones/DistributionZonesTestUtil.java
index 1cdabe81d1..729f15d143 100644
--- a/modules/distribution-zones/src/testFixtures/java/org/apache/ignite/internal/distributionzones/DistributionZonesTestUtil.java
+++ b/modules/distribution-zones/src/testFixtures/java/org/apache/ignite/internal/distributionzones/DistributionZonesTestUtil.java
@@ -17,6 +17,7 @@
package org.apache.ignite.internal.distributionzones;
+import static java.util.concurrent.CompletableFuture.completedFuture;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.DEFAULT_FILTER;
import static org.apache.ignite.internal.distributionzones.DistributionZoneManager.IMMEDIATE_TIMER_VALUE;
import static org.apache.ignite.internal.distributionzones.DistributionZonesUtil.updateLogicalTopologyAndVersion;
@@ -41,6 +42,11 @@ import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
import java.util.Map;
import java.util.Objects;
@@ -48,9 +54,12 @@ import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutionException;
+import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Consumer;
import java.util.function.Function;
import java.util.stream.Collectors;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.cluster.management.topology.api.LogicalNode;
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters.Builder;
import org.apache.ignite.internal.metastorage.MetaStorageManager;
@@ -486,4 +495,29 @@ public class DistributionZonesTestUtil {
assertThat(dataNodes, is(expectedValueNames));
}
}
+
+ /**
+ * Creates CatalogService mock.
+ */
+ public static CatalogManager mockCatalog() {
+ AtomicInteger idGen = new AtomicInteger(0);
+
+ CatalogManager catalogManager = mock(CatalogManager.class);
+ when(catalogManager.createDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.alterDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.renameDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.dropDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.zone(anyString(), anyLong())).thenAnswer(invocation -> zoneDescriptorMock(idGen));
+
+ return catalogManager;
+ }
+
+ private static CatalogZoneDescriptor zoneDescriptorMock(AtomicInteger idGen) {
+ int zoneId = idGen.incrementAndGet();
+
+ CatalogZoneDescriptor descriptor = mock(CatalogZoneDescriptor.class);
+ when(descriptor.id()).thenReturn(zoneId);
+
+ return descriptor;
+ }
}
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
index d062b4caac..01c6b7c419 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/configuration/storage/ItRebalanceDistributedTest.java
@@ -796,6 +796,7 @@ public class ItRebalanceDistributedTest {
distributionZoneManager = new DistributionZoneManager(
zonesCfg,
tablesCfg,
+ catalogManager,
metaStorageManager,
logicalTopologyService,
vaultManager,
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/distribution/zones/ItIgniteDistributionZoneManagerNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/distribution/zones/ItIgniteDistributionZoneManagerNodeRestartTest.java
index daff409fc2..3b2174a6e0 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/distribution/zones/ItIgniteDistributionZoneManagerNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/distribution/zones/ItIgniteDistributionZoneManagerNodeRestartTest.java
@@ -67,6 +67,7 @@ import org.apache.ignite.internal.configuration.validation.ConfigurationValidato
import org.apache.ignite.internal.distributionzones.DistributionZoneConfigurationParameters;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.distributionzones.DistributionZoneManager.ZoneState;
+import org.apache.ignite.internal.distributionzones.DistributionZonesTestUtil;
import org.apache.ignite.internal.distributionzones.NodeWithAttributes;
import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.manager.IgniteComponent;
@@ -204,6 +205,7 @@ public class ItIgniteDistributionZoneManagerNodeRestartTest extends BaseIgniteRe
DistributionZoneManager distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfiguration,
+ DistributionZonesTestUtil.mockCatalog(),
metaStorageMgr,
logicalTopologyService,
vault,
diff --git a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
index 7a3c00c0d0..831421988c 100644
--- a/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
+++ b/modules/runner/src/integrationTest/java/org/apache/ignite/internal/runner/app/ItIgniteNodeRestartTest.java
@@ -339,24 +339,25 @@ public class ItIgniteNodeRestartTest extends BaseIgniteRestartTest {
GcConfiguration gcConfig = clusterConfigRegistry.getConfiguration(GcConfiguration.KEY);
+ var clockWaiter = new ClockWaiter("test", hybridClock);
+
+ var catalogManager = new CatalogManagerImpl(
+ new UpdateLogImpl(metaStorageMgr),
+ clockWaiter
+ );
+
SchemaManager schemaManager = new SchemaManager(registry, tablesConfig, metaStorageMgr);
DistributionZoneManager distributionZoneManager = new DistributionZoneManager(
zonesConfig,
tablesConfig,
+ catalogManager,
metaStorageMgr,
logicalTopologyService,
vault,
name
);
- var clockWaiter = new ClockWaiter("test", hybridClock);
-
- var catalogManager = new CatalogManagerImpl(
- new UpdateLogImpl(metaStorageMgr),
- clockWaiter
- );
-
TableManager tableManager = new TableManager(
name,
registry,
diff --git a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
index de5e7e783a..94c63b5d6e 100644
--- a/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
+++ b/modules/runner/src/main/java/org/apache/ignite/internal/app/IgniteImpl.java
@@ -498,9 +498,20 @@ public class IgniteImpl implements Ignite {
schemaManager = new SchemaManager(registry, tablesConfig, metaStorageMgr);
+ SchemaSynchronizationConfiguration schemaSyncConfig = clusterConfigRegistry.getConfiguration(
+ SchemaSynchronizationConfiguration.KEY
+ );
+
+ catalogManager = new CatalogManagerImpl(
+ new UpdateLogImpl(metaStorageMgr),
+ clockWaiter,
+ () -> schemaSyncConfig.delayDuration().value()
+ );
+
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
tablesConfig,
+ catalogManager,
metaStorageMgr,
logicalTopologyService,
vaultMgr,
@@ -511,16 +522,6 @@ public class IgniteImpl implements Ignite {
outgoingSnapshotsManager = new OutgoingSnapshotsManager(clusterSvc.messagingService());
- SchemaSynchronizationConfiguration schemaSyncConfig = clusterConfigRegistry.getConfiguration(
- SchemaSynchronizationConfiguration.KEY
- );
-
- catalogManager = new CatalogManagerImpl(
- new UpdateLogImpl(metaStorageMgr),
- clockWaiter,
- () -> schemaSyncConfig.delayDuration().value()
- );
-
distributedTblMgr = new TableManager(
name,
registry,
diff --git a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
index 912c66fe3b..a7a9b7f78e 100644
--- a/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
+++ b/modules/sql-engine/src/main/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerWrapper.java
@@ -28,19 +28,13 @@ import org.apache.ignite.internal.index.IndexManager;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterColumnCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableAddCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterTableDropCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterZoneRenameCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.AlterZoneSetCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateTableCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateZoneCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DdlCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropIndexCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropTableCommand;
-import org.apache.ignite.internal.sql.engine.prepare.ddl.DropZoneCommand;
import org.apache.ignite.internal.storage.DataStorageManager;
import org.apache.ignite.internal.table.distributed.TableManager;
-import org.apache.ignite.lang.DistributionZoneAlreadyExistsException;
-import org.apache.ignite.lang.DistributionZoneNotFoundException;
import org.apache.ignite.lang.IndexAlreadyExistsException;
import org.apache.ignite.lang.IndexNotFoundException;
import org.apache.ignite.lang.TableAlreadyExistsException;
@@ -120,34 +114,6 @@ public class DdlCommandHandlerWrapper extends DdlCommandHandler {
.thenCompose(res -> catalogManager.dropIndex(DdlToCatalogCommandConverter.convert((DropIndexCommand) cmd))
.handle(handleModificationResult(((DropIndexCommand) cmd).ifNotExists(), IndexNotFoundException.class))
);
- } else if (cmd instanceof CreateZoneCommand) {
- CreateZoneCommand zoneCommand = (CreateZoneCommand) cmd;
-
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.createDistributionZone(DdlToCatalogCommandConverter.convert(zoneCommand))
- .handle(handleModificationResult(zoneCommand.ifNotExists(), DistributionZoneAlreadyExistsException.class))
- );
- } else if (cmd instanceof DropZoneCommand) {
- DropZoneCommand zoneCommand = (DropZoneCommand) cmd;
-
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.dropDistributionZone(DdlToCatalogCommandConverter.convert(zoneCommand))
- .handle(handleModificationResult(zoneCommand.ifExists(), DistributionZoneNotFoundException.class))
- );
- } else if (cmd instanceof AlterZoneRenameCommand) {
- AlterZoneRenameCommand zoneCommand = (AlterZoneRenameCommand) cmd;
-
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.renameDistributionZone(DdlToCatalogCommandConverter.convert(zoneCommand))
- .handle(handleModificationResult(zoneCommand.ifExists(), DistributionZoneNotFoundException.class))
- );
- } else if (cmd instanceof AlterZoneSetCommand) {
- AlterZoneSetCommand zoneCommand = (AlterZoneSetCommand) cmd;
-
- return ddlCommandFuture
- .thenCompose(res -> catalogManager.alterDistributionZone(DdlToCatalogCommandConverter.convert(zoneCommand))
- .handle(handleModificationResult(zoneCommand.ifExists(), DistributionZoneNotFoundException.class))
- );
}
return ddlCommandFuture;
diff --git a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
index 0c42f9fe49..b9e810ecd0 100644
--- a/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
+++ b/modules/sql-engine/src/test/java/org/apache/ignite/internal/sql/engine/exec/ddl/DdlCommandHandlerExceptionHandlingTest.java
@@ -17,23 +17,26 @@
package org.apache.ignite.internal.sql.engine.exec.ddl;
+import static java.util.concurrent.CompletableFuture.completedFuture;
import static org.apache.ignite.configuration.annotation.ConfigurationType.DISTRIBUTED;
import static org.apache.ignite.internal.testframework.matchers.CompletableFutureMatcher.willCompleteSuccessfully;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyLong;
+import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
-import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
-import org.apache.ignite.configuration.NamedConfigurationTree;
-import org.apache.ignite.configuration.NamedListView;
+import org.apache.ignite.internal.catalog.CatalogManager;
+import org.apache.ignite.internal.catalog.descriptors.CatalogZoneDescriptor;
import org.apache.ignite.internal.configuration.ConfigurationRegistry;
import org.apache.ignite.internal.configuration.ConfigurationTreeGenerator;
import org.apache.ignite.internal.configuration.storage.TestConfigurationStorage;
@@ -42,9 +45,6 @@ import org.apache.ignite.internal.distributionzones.DistributionZoneConfiguratio
import org.apache.ignite.internal.distributionzones.DistributionZoneManager;
import org.apache.ignite.internal.distributionzones.configuration.DistributionZonesConfiguration;
import org.apache.ignite.internal.index.IndexManager;
-import org.apache.ignite.internal.schema.configuration.TableChange;
-import org.apache.ignite.internal.schema.configuration.TableConfiguration;
-import org.apache.ignite.internal.schema.configuration.TableView;
import org.apache.ignite.internal.sql.engine.prepare.ddl.CreateZoneCommand;
import org.apache.ignite.internal.sql.engine.prepare.ddl.DropZoneCommand;
import org.apache.ignite.internal.storage.DataStorageManager;
@@ -106,17 +106,19 @@ public class DdlCommandHandlerExceptionHandlingTest extends IgniteAbstractTest {
DistributionZonesConfiguration zonesConfiguration = registry.getConfiguration(DistributionZonesConfiguration.KEY);
- NamedConfigurationTree<TableConfiguration, TableView, TableChange> tables = mock(NamedConfigurationTree.class);
-
- NamedListView<TableView> value = mock(NamedListView.class);
-
- when(tables.value()).thenReturn(value);
-
- when(value.namedListKeys()).thenReturn(new ArrayList<>());
+ CatalogManager catalogManager = mock(CatalogManager.class);
+ when(catalogManager.createDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.alterDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.renameDistributionZone(any())).thenReturn(completedFuture(null));
+ when(catalogManager.dropDistributionZone(any())).thenReturn(completedFuture(null));
+ CatalogZoneDescriptor desc = mock(CatalogZoneDescriptor.class);
+ when(catalogManager.zone(anyString(), anyLong())).thenReturn(desc);
+ when(desc.id()).thenReturn(42);
distributionZoneManager = new DistributionZoneManager(
zonesConfiguration,
null,
+ catalogManager,
null,
null,
null,