You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "jpisaac (via GitHub)" <gi...@apache.org> on 2025/01/21 20:50:19 UTC
[PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
jpisaac opened a new pull request, #2060:
URL: https://github.com/apache/phoenix/pull/2060
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1936671200
##########
phoenix-core-client/src/main/java/org/apache/phoenix/schema/PTableImpl.java:
##########
@@ -2516,6 +2479,41 @@ public Map<PTableKey, Long> getAncestorLastDDLTimestampMap() {
return ancestorLastDDLTimestampMap;
}
+ private static PColumn getCompatibleColumnTypeForClientVersion(PColumn column, String schemaName, String tableName, String columnName, long clientVersion) {
+ LOGGER.info("getCompatibleColumnTypeForVersion for column Name {}.{}, {}, version{} ",schemaName, tableName, columnName, clientVersion);
+ if ((column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.ROW_KEY_MATCHER )
+ || column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.PARTITION_START_KEY)
+ || column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.PARTITION_END_KEY))
+ && clientVersion < MetaDataProtocol.MIN_VERSION_ALLOW_VBE_COLUMNS) {
+ PColumn backwardCompatibleColumn =
+ new PColumnImpl(column.getName(),
+ column.getFamilyName(),
+ PVarbinary.INSTANCE,
+ column.getMaxLength(),
+ column.getScale(),
+ column.isNullable(),
+ column.getPosition(),
+ column.getSortOrder(),
+ column.getArraySize(),
+ column.getViewConstant(),
+ column.isViewReferenced(),
+ column.getExpressionStr(),
+ column.isRowTimestamp(),
+ column.isDynamic(),
+ column.getColumnQualifierBytes(),
+ column.getTimestamp());
+
+ LOGGER.info("Adapting field type to {} from {}, MIN_CATALOG_TS {}, CLIENT_VERSION {}, MIN_ALLOWED_VERSION {}",
+ backwardCompatibleColumn.getDataType().getSqlTypeName(),
+ column.getDataType().getSqlTypeName(),
+ MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP,
+ clientVersion,
+ MetaDataProtocol.MIN_VERSION_ALLOW_VBE_COLUMNS);
+ return backwardCompatibleColumn;
Review Comment:
We typically do not pass nulls in the List<PColumn> and the downstream code paths are not explicitly checking for nulls, thus gives a NullPointer exception.
If we skip the column all together then it gives another exception, I guess since there is a column count mismatch OR and some other logic error (haven't had a chance to debug)
Error: ERROR 203 (22005): Type mismatch. expected: [UNSIGNED_TINYINT] but was: BIGINT at SQLINDEXTYPE argument 1 (state=22005,code=203)
org.apache.phoenix.schema.ArgumentTypeMismatchException: ERROR 203 (22005): Type mismatch. expected: [UNSIGNED_TINYINT] but was: BIGINT at SQLINDEXTYPE argument 1
at org.apache.phoenix.parse.FunctionParseNode.validateFunctionArguement(FunctionParseNode.java:215)
at org.apache.phoenix.parse.FunctionParseNode.validate(FunctionParseNode.java:194)
at org.apache.phoenix.compile.ExpressionCompiler.visitLeave(ExpressionCompiler.java:327)
at org.apache.phoenix.compile.ProjectionCompiler$SelectClauseVisitor.visitLeave(ProjectionCompiler.java:779)
at org.apache.phoenix.compile.ProjectionCompiler$SelectClauseVisitor.visitLeave(ProjectionCompiler.java:659)
at org.apache.phoenix.parse.FunctionParseNode.accept(FunctionParseNode.java:87)
at org.apache.phoenix.compile.ProjectionCompiler.compile(ProjectionCompiler.java:452)
at org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:755)
at org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:668)
at org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:246)
at org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:177)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:722)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:685)
at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:360)
at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:341)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:341)
at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:327)
at org.apache.phoenix.jdbc.PhoenixPreparedStatement.executeQuery(PhoenixPreparedStatement.java:195)
at org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.getTables(PhoenixDatabaseMetaData.java:1425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sqlline.Reflector.invoke(Reflector.java:75)
at sqlline.Commands.metadata(Commands.java:202)
at sqlline.Commands.tables(Commands.java:340)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
at sqlline.SqlLine.dispatch(SqlLine.java:643)
at sqlline.SqlLine.begin(SqlLine.java:510)
at sqlline.SqlLine.start(SqlLine.java:233)
at sqlline.SqlLine.main(SqlLine.java:175)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924499824
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewTTLIT.java:
##########
@@ -2347,14 +2478,178 @@ protected void testMajorCompactWithVariousTenantIdTypesAndRegions(PDataType tena
}
+ protected void testMajorCompactTenantViewsWithVariousPKTypesAndSortOrder() throws Exception {
+ try {
+ List<PDataType[]> testCases = getTestCases();
+ SortOrder[][] sortOrders = getSortOrders();
+ for (PDataType[] aCase : testCases) {
+ for (SortOrder[] sortOrder : sortOrders) {
+ runTenantViewsWithVariousPKTypes(aCase, sortOrder);
+ }
+ }
+ } catch (Exception e) {
+ LOGGER.info(LogUtil.getCallerStackTrace());
+ LOGGER.error(e.getMessage());
+ }
+
+ }
+
+ /**
+ * Test special case:
+ * Test with various PK Types and SortOrders
+ * This can occur when the TENANT_ID and global view PARTITION_KEY overlap.
+ * @throws Exception
+ */
+
+ private void runTenantViewsWithVariousPKTypes(PDataType[] pkDataTypes, SortOrder[] sortOrders) throws Exception {
+ // View TTL is set in seconds (for e.g 10 secs)
+ resetEnvironmentEdgeManager();
+ int viewTTL = VIEW_TTL_10_SECS;
+ // Define the test schema.
+ // 1. Table with columns => (ORG_ID, KP, COL1, COL2, COL3), PK => (ORG_ID, KP)
+ // 2. GlobalView with columns => (ID, COL4, COL5, COL6), PK => (ID)
+ // 3. Tenant with columns => (ZID, COL7, COL8, COL9), PK => (ZID)
+ final SchemaBuilder schemaBuilder = new SchemaBuilder(getUrl());
+
+ TableOptions
+ tableOptions = TableOptions.withDefaults();
+ tableOptions.setTableProps("");
+ tableOptions.setTableProps("COLUMN_ENCODED_BYTES=0,MULTI_TENANT=true,DEFAULT_COLUMN_FAMILY='0'");
+ tableOptions.setTablePKColumns(Arrays.asList("OID", "KP"));
+ tableOptions.setTablePKColumnTypes(Arrays.asList("CHAR(15)", "CHAR(3)"));
+
+ DataOptions dataOptions = DataOptions.withDefaults();
+ dataOptions.setTenantViewName("Z01");
+ dataOptions.setKeyPrefix("Z01");
+
+ GlobalViewOptions
+ globalViewOptions = GlobalViewOptions.withDefaults();
+ String[] globalViewPKNames = new String[] { "ID1", "ID2", "ID3" };
+ globalViewOptions.setGlobalViewPKColumns(asList(globalViewPKNames));
+ globalViewOptions.setGlobalViewPKColumnTypes(
+ asList(
+ pkDataTypes[0].getSqlTypeName(),
+ pkDataTypes[1].getSqlTypeName(),
+ pkDataTypes[2].getSqlTypeName()
+ )
+ );
+ globalViewOptions.setGlobalViewPKColumnSort(asList(
+ sortOrders[0].name(),
+ sortOrders[1].name(),
+ sortOrders[2].name()
+ ));
+
+ // View TTL is set to 10s => 10000 ms
+ //globalViewOptions.setTableProps(String.format("TTL=%d", viewTTL));
Review Comment:
Remove commented
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924499217
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -320,12 +344,16 @@ private Pair<String, byte[]> createTenantView(boolean extendPK, int partition,
"CONSTRAINT pk PRIMARY KEY (ZID)) " + "AS SELECT * FROM %s %s %s";
String VIEW_WO_PK_TEMPLATE = "CREATE VIEW IF NOT EXISTS %s AS SELECT * from %s %s";
if (extendPK) {
- cstmt.execute(
+ String viewWithPKSQL =
String.format(VIEW_WITH_PK_TEMPLATE, tenantViewName, globalViewName,
- getWhereClause(pkNames, pkTypes), tenantViewOptions));
+ getWhereClause(pkNames, pkTypes), tenantViewOptions);
+ LOGGER.info(String.format("Created tenant view (WITH_PK) %s", viewWithPKSQL));
Review Comment:
nit: `LOGGER.info("Created tenant view (WITH_PK) {}", viewWithPKSQL)`
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -271,17 +291,21 @@ private Pair<String, byte[]> createGlobalView(String tableName, int partition,
"CONSTRAINT pk PRIMARY KEY (ID1 %s, ID2 %s, ID3 %s, ROW_ID)) " +
"AS SELECT * FROM %s WHERE KP = '%s'";
- cstmt.execute(String.format(VIEW_TEMPLATE, globalViewName, pkType1Str, pkType2Str,
+ String globalViewSQL = String.format(VIEW_TEMPLATE, globalViewName, pkType1Str, pkType2Str,
pkType3Str, pkOrders[0].name(), pkOrders[1].name(), pkOrders[2].name(),
- baseTableName, partitionName));
+ baseTableName, partitionName);
+ LOGGER.info(String.format("Created global view %s", globalViewSQL));
Review Comment:
nit: `LOGGER.info("Created global view {}", globalViewSQL)`
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -320,12 +344,16 @@ private Pair<String, byte[]> createTenantView(boolean extendPK, int partition,
"CONSTRAINT pk PRIMARY KEY (ZID)) " + "AS SELECT * FROM %s %s %s";
String VIEW_WO_PK_TEMPLATE = "CREATE VIEW IF NOT EXISTS %s AS SELECT * from %s %s";
if (extendPK) {
- cstmt.execute(
+ String viewWithPKSQL =
String.format(VIEW_WITH_PK_TEMPLATE, tenantViewName, globalViewName,
- getWhereClause(pkNames, pkTypes), tenantViewOptions));
+ getWhereClause(pkNames, pkTypes), tenantViewOptions);
+ LOGGER.info(String.format("Created tenant view (WITH_PK) %s", viewWithPKSQL));
+ cstmt.execute(viewWithPKSQL);
} else {
- cstmt.execute(String.format(VIEW_WO_PK_TEMPLATE, tenantViewName, globalViewName,
- tenantViewOptions));
+ String viewWithoutPKSQL = String.format(VIEW_WO_PK_TEMPLATE, tenantViewName, globalViewName,
+ tenantViewOptions);
+ LOGGER.info(String.format("Created tenant view ((WO_PK)) %s", viewWithoutPKSQL));
Review Comment:
same here
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -271,17 +291,21 @@ private Pair<String, byte[]> createGlobalView(String tableName, int partition,
"CONSTRAINT pk PRIMARY KEY (ID1 %s, ID2 %s, ID3 %s, ROW_ID)) " +
"AS SELECT * FROM %s WHERE KP = '%s'";
- cstmt.execute(String.format(VIEW_TEMPLATE, globalViewName, pkType1Str, pkType2Str,
+ String globalViewSQL = String.format(VIEW_TEMPLATE, globalViewName, pkType1Str, pkType2Str,
pkType3Str, pkOrders[0].name(), pkOrders[1].name(), pkOrders[2].name(),
- baseTableName, partitionName));
+ baseTableName, partitionName);
+ LOGGER.info(String.format("Created global view %s", globalViewSQL));
+ cstmt.execute(globalViewSQL);
if (hasGlobalViewIndexes) {
String indexNamePrefix = String.format("G%s", partition);
String
GLOBAL_INDEX_TEMPLATE =
"CREATE INDEX IF NOT EXISTS %s_COL2_INDEX ON %s (COL2) " +
"INCLUDE(SYSTEM_MODSTAMP)";
- cstmt.execute(
- String.format(GLOBAL_INDEX_TEMPLATE, indexNamePrefix, globalViewName));
+ String globalViewIndexSQL =
+ String.format(GLOBAL_INDEX_TEMPLATE, indexNamePrefix, globalViewName);
+ LOGGER.info(String.format("Created global view index %s", globalViewIndexSQL));
Review Comment:
same here, String.format not needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924499261
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewTTLIT.java:
##########
@@ -164,6 +181,120 @@ public void testResetServerCache() {
}
}
+ private SortOrder[][] getSortOrders() {
+ SortOrder[][]
+ sortOrders =
+ new SortOrder[][] {
+ { SortOrder.ASC, SortOrder.ASC, SortOrder.ASC },
+ { SortOrder.ASC, SortOrder.ASC, SortOrder.DESC },
+ { SortOrder.ASC, SortOrder.DESC, SortOrder.ASC },
+ { SortOrder.ASC, SortOrder.DESC, SortOrder.DESC },
+ { SortOrder.DESC, SortOrder.ASC, SortOrder.ASC },
+ { SortOrder.DESC, SortOrder.ASC, SortOrder.DESC },
+ { SortOrder.DESC, SortOrder.DESC, SortOrder.ASC },
+ { SortOrder.DESC, SortOrder.DESC, SortOrder.DESC }
+ };
+ return sortOrders;
+ }
+
+
+ private List<PDataType[]> getTestCases() {
+
+ List<PDataType[]> testCases = new ArrayList<>();
+ // Test Case 1: PK1 = Integer, PK2 = Integer, PK3 = Integer
+ testCases.add(new PDataType[] { PInteger.INSTANCE, PInteger.INSTANCE, PInteger.INSTANCE });
+ // Test Case 2: PK1 = Long, PK2 = Long, PK3 = Long
+ testCases.add(new PDataType[] { PLong.INSTANCE, PLong.INSTANCE, PLong.INSTANCE });
+ // Test Case 3: PK1 = Timestamp, PK2 = Timestamp, PK3 = Timestamp
+ testCases.add(
+ new PDataType[] { PTimestamp.INSTANCE, PTimestamp.INSTANCE, PTimestamp.INSTANCE });
+ // Test Case 4: PK1 = Char, PK2 = Char, PK3 = Char
+ testCases.add(new PDataType[] { PChar.INSTANCE, PChar.INSTANCE, PChar.INSTANCE });
+ // Test Case 5: PK1 = Decimal, PK2 = Decimal, PK3 = Integer
+ // last PK cannot be of variable length when creating a view on top of it
+ testCases.add(new PDataType[] { PDecimal.INSTANCE, PDecimal.INSTANCE, PInteger.INSTANCE });
+ // Test Case 6: PK1 = Date, PK2 = Date, PK3 = Date
+ testCases.add(new PDataType[] { PDate.INSTANCE, PDate.INSTANCE, PDate.INSTANCE });
+ // Test Case 7: PK1 = Varchar, PK2 = Varchar, PK3 = Integer
+ // last PK cannot be of variable length when creating a view on top of it
+ testCases.add(new PDataType[] { PVarchar.INSTANCE, PVarchar.INSTANCE, PInteger.INSTANCE });
+
+ // Test Case 8: PK1 = VARBINARY_ENCODED, PK2 = Varchar, PK3 = VARBINARY_ENCODED
+ testCases.add(new PDataType[] { PVarbinaryEncoded.INSTANCE, PVarchar.INSTANCE, PInteger.INSTANCE });
+ return testCases;
+ }
+
+ private String getWhereClause(String[] pkNames, PDataType[] testPKTypes) {
+
+ StringBuilder builder = new StringBuilder("WHERE ");
+ Random rnd = new Random();
+
+ for (int b = 0; b < testPKTypes.length; b++) {
+ if (b > 0) builder.append(" AND ");
+ switch (testPKTypes[b].getSqlType()) {
+ case Types.VARCHAR: {
+ // pkTypeStr = "VARCHAR(25)";
+ builder.append(pkNames[b]).append(" = ").append("'")
+ .append(RandomStringUtils.randomAlphanumeric(25)).append("'");
+ break;
+ }
+ case Types.CHAR: {
+ //pkTypeStr = "CHAR(15)";
+ builder.append(pkNames[b]).append(" = ").append("'")
+ .append(RandomStringUtils.randomAlphanumeric(15)).append("'");
+ break;
+ }
+ case Types.DECIMAL:
+ //pkTypeStr = "DECIMAL(8,2)";
+ builder.append(pkNames[b]).append(" = ").append(rnd.nextDouble());
+ break;
+ case Types.INTEGER:
+ //pkTypeStr = "INTEGER";
+ builder.append(pkNames[b]).append(" = ").append(rnd.nextInt(500000));
+ break;
+ case Types.BIGINT:
+ //pkTypeStr = "BIGINT";
+ builder.append(pkNames[b]).append(" = ").append(rnd.nextLong());
+ break;
+ case Types.DATE:
+ //pkTypeStr = "DATE";
+ builder.append(pkNames[b]).append(" = ")
+ .append(" TO_DATE('2022-03-21T15:03:57+00:00') ");
+ break;
+ case Types.TIMESTAMP:
+ //pkTypeStr = "TIMESTAMP";
+ builder.append(pkNames[b]).append(" = ")
+ .append(" TO_TIMESTAMP('2019-10-27T16:17:57+00:00') ");
+ break;
+ case Types.VARBINARY:
+ // pkTypeStr = "VARBINARY";
+ case 9000:
+ // pkTypeStr = "VARBINARY_ENCODED";
+ byte[] varBytes = ByteUtil.concat(
+ RandomStringUtils.randomAlphanumeric(25).getBytes(),
+ Bytes.toBytes(rnd.nextInt(50000)),
+ Bytes.toBytes(Math.floor(rnd.nextInt(50000) * rnd.nextDouble())));
+ LOGGER.info("***************************************************************************");
Review Comment:
Remove logging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1936374592
##########
phoenix-core-client/src/main/java/org/apache/phoenix/schema/PTableImpl.java:
##########
@@ -2516,6 +2479,41 @@ public Map<PTableKey, Long> getAncestorLastDDLTimestampMap() {
return ancestorLastDDLTimestampMap;
}
+ private static PColumn getCompatibleColumnTypeForClientVersion(PColumn column, String schemaName, String tableName, String columnName, long clientVersion) {
+ LOGGER.info("getCompatibleColumnTypeForVersion for column Name {}.{}, {}, version{} ",schemaName, tableName, columnName, clientVersion);
+ if ((column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.ROW_KEY_MATCHER )
+ || column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.PARTITION_START_KEY)
+ || column.getName().getString().equalsIgnoreCase(PhoenixDatabaseMetaData.PARTITION_END_KEY))
+ && clientVersion < MetaDataProtocol.MIN_VERSION_ALLOW_VBE_COLUMNS) {
+ PColumn backwardCompatibleColumn =
+ new PColumnImpl(column.getName(),
+ column.getFamilyName(),
+ PVarbinary.INSTANCE,
+ column.getMaxLength(),
+ column.getScale(),
+ column.isNullable(),
+ column.getPosition(),
+ column.getSortOrder(),
+ column.getArraySize(),
+ column.getViewConstant(),
+ column.isViewReferenced(),
+ column.getExpressionStr(),
+ column.isRowTimestamp(),
+ column.isDynamic(),
+ column.getColumnQualifierBytes(),
+ column.getTimestamp());
+
+ LOGGER.info("Adapting field type to {} from {}, MIN_CATALOG_TS {}, CLIENT_VERSION {}, MIN_ALLOWED_VERSION {}",
+ backwardCompatibleColumn.getDataType().getSqlTypeName(),
+ column.getDataType().getSqlTypeName(),
+ MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP,
+ clientVersion,
+ MetaDataProtocol.MIN_VERSION_ALLOW_VBE_COLUMNS);
+ return backwardCompatibleColumn;
Review Comment:
Instead of returning VARBINARY column for ROW_KEY_MATCHER, can't we return null here i.e. no column?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "kadirozde (via GitHub)" <gi...@apache.org>.
kadirozde commented on PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#issuecomment-2631883008
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac merged PR #2060:
URL: https://github.com/apache/phoenix/pull/2060
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#issuecomment-2610964948
> +1, pending build results from https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-2060/2/
The backward compatibility test failures are due to VARBINARY_ENCODED type not being in other branches (for e.g 5.1.*) @virajjasani
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924500062
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewTTLIT.java:
##########
@@ -2347,14 +2478,178 @@ protected void testMajorCompactWithVariousTenantIdTypesAndRegions(PDataType tena
}
+ protected void testMajorCompactTenantViewsWithVariousPKTypesAndSortOrder() throws Exception {
+ try {
+ List<PDataType[]> testCases = getTestCases();
+ SortOrder[][] sortOrders = getSortOrders();
+ for (PDataType[] aCase : testCases) {
+ for (SortOrder[] sortOrder : sortOrders) {
+ runTenantViewsWithVariousPKTypes(aCase, sortOrder);
+ }
+ }
+ } catch (Exception e) {
+ LOGGER.info(LogUtil.getCallerStackTrace());
+ LOGGER.error(e.getMessage());
+ }
+
+ }
+
+ /**
+ * Test special case:
+ * Test with various PK Types and SortOrders
+ * This can occur when the TENANT_ID and global view PARTITION_KEY overlap.
+ * @throws Exception
+ */
+
+ private void runTenantViewsWithVariousPKTypes(PDataType[] pkDataTypes, SortOrder[] sortOrders) throws Exception {
+ // View TTL is set in seconds (for e.g 10 secs)
+ resetEnvironmentEdgeManager();
+ int viewTTL = VIEW_TTL_10_SECS;
+ // Define the test schema.
+ // 1. Table with columns => (ORG_ID, KP, COL1, COL2, COL3), PK => (ORG_ID, KP)
+ // 2. GlobalView with columns => (ID, COL4, COL5, COL6), PK => (ID)
+ // 3. Tenant with columns => (ZID, COL7, COL8, COL9), PK => (ZID)
+ final SchemaBuilder schemaBuilder = new SchemaBuilder(getUrl());
+
+ TableOptions
+ tableOptions = TableOptions.withDefaults();
+ tableOptions.setTableProps("");
+ tableOptions.setTableProps("COLUMN_ENCODED_BYTES=0,MULTI_TENANT=true,DEFAULT_COLUMN_FAMILY='0'");
+ tableOptions.setTablePKColumns(Arrays.asList("OID", "KP"));
+ tableOptions.setTablePKColumnTypes(Arrays.asList("CHAR(15)", "CHAR(3)"));
+
+ DataOptions dataOptions = DataOptions.withDefaults();
+ dataOptions.setTenantViewName("Z01");
+ dataOptions.setKeyPrefix("Z01");
+
+ GlobalViewOptions
+ globalViewOptions = GlobalViewOptions.withDefaults();
+ String[] globalViewPKNames = new String[] { "ID1", "ID2", "ID3" };
+ globalViewOptions.setGlobalViewPKColumns(asList(globalViewPKNames));
+ globalViewOptions.setGlobalViewPKColumnTypes(
+ asList(
+ pkDataTypes[0].getSqlTypeName(),
+ pkDataTypes[1].getSqlTypeName(),
+ pkDataTypes[2].getSqlTypeName()
+ )
+ );
+ globalViewOptions.setGlobalViewPKColumnSort(asList(
+ sortOrders[0].name(),
+ sortOrders[1].name(),
+ sortOrders[2].name()
+ ));
+
+ // View TTL is set to 10s => 10000 ms
+ //globalViewOptions.setTableProps(String.format("TTL=%d", viewTTL));
+
+
+ TenantViewOptions
+ tenantViewOptions = TenantViewOptions.withDefaults();
+ tenantViewOptions.setTenantViewCondition(String.format(
+ "SELECT * FROM %s.%s %s",
+ dataOptions.getSchemaName(), dataOptions.getGlobalViewName(),
+ getWhereClause(globalViewPKNames, pkDataTypes)));
+ tenantViewOptions.setTableProps(String.format("TTL=%d", viewTTL));
+
+// OtherOptions otherOptions = new OtherOptions();
+// otherOptions.setTableCFs(asList(null, null, null));
Review Comment:
Remove commented
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924497825
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -183,6 +194,15 @@ private Object getData(PDataType type) {
case Types.TIMESTAMP:
//pkTypeStr = "TIMESTAMP";
return new Timestamp(System.currentTimeMillis() + rnd.nextInt(50000));
+ case Types.VARBINARY:
+ // pkTypeStr = "VARBINARY";
+ case 9000:
Review Comment:
same here: `PDataType.VARBINARY_ENCODED_TYPE`
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseRowKeyMatcherTestIT.java:
##########
@@ -153,6 +158,12 @@ private String getType(PDataType pkType) {
case Types.TIMESTAMP:
pkTypeStr = "TIMESTAMP";
break;
+ case Types.VARBINARY:
+ pkTypeStr = "VARBINARY";
+ break;
+ case 9000:
Review Comment:
nit: `PDataType.VARBINARY_ENCODED_TYPE`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on code in PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#discussion_r1924500434
##########
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseViewTTLIT.java:
##########
@@ -2347,14 +2478,178 @@ protected void testMajorCompactWithVariousTenantIdTypesAndRegions(PDataType tena
}
+ protected void testMajorCompactTenantViewsWithVariousPKTypesAndSortOrder() throws Exception {
+ try {
+ List<PDataType[]> testCases = getTestCases();
+ SortOrder[][] sortOrders = getSortOrders();
+ for (PDataType[] aCase : testCases) {
+ for (SortOrder[] sortOrder : sortOrders) {
+ runTenantViewsWithVariousPKTypes(aCase, sortOrder);
+ }
+ }
+ } catch (Exception e) {
+ LOGGER.info(LogUtil.getCallerStackTrace());
+ LOGGER.error(e.getMessage());
+ }
+
+ }
+
+ /**
+ * Test special case:
+ * Test with various PK Types and SortOrders
+ * This can occur when the TENANT_ID and global view PARTITION_KEY overlap.
+ * @throws Exception
+ */
+
+ private void runTenantViewsWithVariousPKTypes(PDataType[] pkDataTypes, SortOrder[] sortOrders) throws Exception {
+ // View TTL is set in seconds (for e.g 10 secs)
+ resetEnvironmentEdgeManager();
+ int viewTTL = VIEW_TTL_10_SECS;
+ // Define the test schema.
+ // 1. Table with columns => (ORG_ID, KP, COL1, COL2, COL3), PK => (ORG_ID, KP)
+ // 2. GlobalView with columns => (ID, COL4, COL5, COL6), PK => (ID)
Review Comment:
Update comments with actual column names
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7507 Update ROW_KEY_MATCHER column type to be VARBINARY_ENCODED [phoenix]
Posted by "jpisaac (via GitHub)" <gi...@apache.org>.
jpisaac commented on PR #2060:
URL: https://github.com/apache/phoenix/pull/2060#issuecomment-2628401495
@tkhurana @virajjasani FYI updated with review comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org