You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Xinyi Yan (Jira)" <ji...@apache.org> on 2020/09/02 05:22:00 UTC
[jira] [Commented] (PHOENIX-5712) Got SYSCAT ILLEGAL_DATA
exception after created tenant index on view
[ https://issues.apache.org/jira/browse/PHOENIX-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188985#comment-17188985 ]
Xinyi Yan commented on PHOENIX-5712:
------------------------------------
I just noticed that we have *phoenix.index.longViewIndex.enabled(true)* config for backward compatibility. With this config and 4.x server/client, I created a view index and ran the select query *VIEW_INDEX_ID* and *VIEW_INDEX_ID_DATA_TYPE*. I had no issue getting values; however, the new client read old client created view index metadata failed with msg (SQLException(ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 3). The old client also had a problem running a select query when it hit the index path on the new client created view index, but the 4.x client had no issue. To summarize this behavior(after 4.x and 4.14.3 client created two-view indexes):
|| ||4.x client||4.14.3 client||
|select *VIEW_INDEX_ID_DATA_TYPE* from syscat |no issue|N/A|
|select *VIEW_INDEX_ID* from syscat *without* 4.14.3 created view index|no issue|N/A|
|select *VIEW_INDEX_ID* from syscat|SQLException(Illegal data. Expected length of at least 8 bytes, but had *3*)|no issue|
|select * from 4.14 created view index|no issue|no issue|
|select * from 4.x created view index|no issue|no issue|
|select index col from 4.14 created view index|no issue|no issue|
|select index col from 4.x created view index|no issue|SQLException(Illegal data. Expected length of at least 8 bytes, but had *2*)|
Timeline and logging:
*NEW CLIENT:*
CREATE TABLE IF NOT EXISTS S.T1 (A INTEGER PRIMARY KEY, B INTEGER) COLUMN_ENCODED_BYTE=0;
CREATE VIEW IF NOT EXISTS S.NEW_CLIENT_VIEW (new_col INTEGER) AS SELECT * FROM S.T1;
CREATE INDEX IF NOT EXISTS NEW_CLIENT_IDX1 ON S.NEW_CLIENT_VIEW (new_col);
SELECT TABLE_NAME,VIEW_INDEX_ID,VIEW_INDEX_ID_DATA_TYPE FROM SYSTEM.CATALOG WHERE VIEW_INDEX_ID IS NOT NULL;
+------------------+----------------+--------------------------+
| TABLE_NAME | VIEW_INDEX_ID | VIEW_INDEX_ID_DATA_TYPE |
+------------------+----------------+--------------------------+
| NEW_CLIENT_IDX1 | -32768 | -5 |
+------------------+----------------+--------------------------+
*OLD CLIENT*:
CREATE TABLE IF NOT EXISTS S.T2 (A INTEGER PRIMARY KEY, B INTEGER) COLUMN_ENCODED_BYTE=0;
CREATE VIEW IF NOT EXISTS S.OLD_CLIENT_VIEW (new_col INTEGER) AS SELECT * FROM S.T2;
CREATE INDEX IF NOT EXISTS OLD_CLIENT_IDX1 ON S.NEW_CLIENT_VIEW (new_col);
SELECT TABLE_NAME,VIEW_INDEX_ID FROM SYSTEM.CATALOG WHERE VIEW_INDEX_ID IS NOT NULL;
+------------------+----------------+
| TABLE_NAME | VIEW_INDEX_ID |
+------------------+----------------+
| NEW_CLIENT_IDX1 | -1 |
| OLD_CLIENT_IDX1 | -32767 |
+------------------+----------------+
SELECT NEW_COL FROM S.OLD_CLIENT_VIEW;
+----------+
| NEW_COL |
+----------+
| 2 |
+----------+
SELECT NEW_COL FROM S.NEW_CLIENT_VIEW;
Error: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 2 (state=22000,code=201)
java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had *2*
at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:522)
at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:195)
at org.apache.phoenix.schema.types.PDataType.checkForSufficientLength(PDataType.java:290)
at org.apache.phoenix.schema.types.PLong$LongCodec.decodeLong(PLong.java:256)
at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:115)
at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:31)
at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:994)
at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:998)
at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:1027)
at org.apache.phoenix.schema.types.PDataType.toStringLiteral(PDataType.java:1100)
at org.apache.phoenix.schema.types.PDataType.toStringLiteral(PDataType.java:1096)
at org.apache.phoenix.iterate.ExplainTable.appendPKColumnValue(ExplainTable.java:212)
at org.apache.phoenix.iterate.ExplainTable.appendScanRow(ExplainTable.java:294)
at org.apache.phoenix.iterate.ExplainTable.appendKeyRanges(ExplainTable.java:307)
at org.apache.phoenix.iterate.ExplainTable.explain(ExplainTable.java:127)
at org.apache.phoenix.iterate.BaseResultIterators.explain(BaseResultIterators.java:1584)
at org.apache.phoenix.iterate.RoundRobinResultIterator.explain(RoundRobinResultIterator.java:154)
at org.apache.phoenix.execute.BaseQueryPlan.getPlanSteps(BaseQueryPlan.java:528)
at org.apache.phoenix.execute.BaseQueryPlan.iterator(BaseQueryPlan.java:376)
at org.apache.phoenix.execute.BaseQueryPlan.iterator(BaseQueryPlan.java:218)
at org.apache.phoenix.execute.BaseQueryPlan.iterator(BaseQueryPlan.java:213)
at org.apache.phoenix.execute.BaseQueryPlan.iterator(BaseQueryPlan.java:208)
at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:342)
at org.apache.phoenix.jdbc.PhoenixStatement$1.call(PhoenixStatement.java:310)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:309)
at org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:302)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1935)
at sqlline.Commands.execute(Commands.java:814)
at sqlline.Commands.sql(Commands.java:754)
at sqlline.SqlLine.dispatch(SqlLine.java:646)
at sqlline.SqlLine.begin(SqlLine.java:510)
at sqlline.SqlLine.start(SqlLine.java:233)
at sqlline.SqlLine.main(SqlLine.java:175)
*NEW CLIENT*
UPSERT INTO S.NEW_CLIENT_VIEW (A,B,NEW_COL) VALUES (1,1,1);
SELECT NEW_COL FROM S.NEW_CLIENT_VIEW;
+----------+
| NEW_COL |
+----------+
| 1 |
+----------+
SELECT NEW_COL FROM S.OLD_CLIENT_VIEW;
+----------+
| NEW_COL |
+----------+
| 2 |
+----------+
SELECT TABLE_NAME, VIEW_INDEX_ID_DATA_TYPE FROM SYSTEM.CATALOG WHERE TABLE_TYPE = 'i' AND COLUMN_COUNT IS NOT NULL;
+------------------+--------------------------+
| TABLE_NAME | VIEW_INDEX_ID_DATA_TYPE |
+------------------+--------------------------+
| NEW_CLIENT_IDX1 | -5 |
| OLD_CLIENT_IDX1 | null |
+------------------+--------------------------+
SELECT TABLE_NAME, VIEW_INDEX_ID FROM SYSTEM.CATALOG WHERE TABLE_TYPE = 'i' AND COLUMN_COUNT IS NOT NULL AND TABLE_NAME='NEW_CLIENT_IDX1';
+------------------+----------------+
| TABLE_NAME | VIEW_INDEX_ID |
+------------------+----------------+
| NEW_CLIENT_IDX1 | -32768 |
+------------------+----------------+
SELECT TABLE_NAME, VIEW_INDEX_ID FROM SYSTEM.CATALOG WHERE TABLE_TYPE = 'i' AND COLUMN_COUNT IS NOT NULL AND TABLE_NAME='OLD_CLIENT_IDX1';
Error: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 3 (state=22000,code=201)
java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had *3*
at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:575)
at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:195)
at org.apache.phoenix.schema.types.PDataType.checkForSufficientLength(PDataType.java:290)
at org.apache.phoenix.schema.types.PLong$LongCodec.decodeLong(PLong.java:256)
at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:115)
at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:31)
at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:1011)
at org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
at org.apache.phoenix.jdbc.PhoenixResultSet.getObject(PhoenixResultSet.java:585)
at sqlline.Rows$Row.<init>(Rows.java:196)
at sqlline.BufferedRows.<init>(BufferedRows.java:38)
at sqlline.SqlLine.print(SqlLine.java:1620)
at sqlline.Commands.execute(Commands.java:825)
at sqlline.Commands.sql(Commands.java:754)
at sqlline.SqlLine.dispatch(SqlLine.java:646)
at sqlline.SqlLine.begin(SqlLine.java:510)
at sqlline.SqlLine.start(SqlLine.java:233)
at sqlline.SqlLine.main(SqlLine.java:175)
Based on my finding, the old client cannot understand the *VIEW_INDEX_ID* if the *VIEW_INDEX_ID_DATA_TYPE*** is a BIGINT instead of a SMALLINT. For the new client, the index view creation did call +getViewIndexDataType+ to get *VIEW_INDEX_ID_DATA_TYPE (*SMALLINT or BIGINT), but I didn't see the logic to get *VIEW_INDEX_ID_DATA_TYPE* while getting syscat metadata on 4.x client. It always assumes the *VIEW_INDEX_ID* is a BIGINT instead of a SMALLINT, so the PDataType checked sufficient length and raised an exception when new client read old client created view index metadata(*VIEW_INDEX_ID* is a 3 bytes immutable bytes ptr) from the syscat. For this reason, overwrite the *VIEW_INDEX_ID_DATA_TYPE* value to a SMALLINT value won't help here.
> Got SYSCAT ILLEGAL_DATA exception after created tenant index on view
> ---------------------------------------------------------------------
>
> Key: PHOENIX-5712
> URL: https://issues.apache.org/jira/browse/PHOENIX-5712
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.15.0
> Reporter: Xinyi Yan
> Priority: Major
> Fix For: 4.16.0
>
> Attachments: t.txt
>
>
> repo
> //create a multi-tenant table on global connection
> CREATE TABLE A (TENANT_ID CHAR(15) NOT NULL, ID CHAR(3) NOT NULL, NUM BIGINT CONSTRAINT PK PRIMARY KEY (TENANT_ID, ID)) MULTI_TENANT = true;
> // create view and index on tenant connection
> CREATE VIEW A_VIEW AS SELECT * FROM A;
> UPSERT INTO A_VIEW (ID, NUM) VALUES ('A', 1);
> CREATE INDEX A_VIEW_INDEX ON A_VIEW (NUM DESC) INCLUDE (ID);
> // qeury data on global connection
> SELECT * RFOM SYSTEM.CATALOG;
> {code:java}
> Error: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 3 (state=22000,code=201)
> java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 3
> at org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:559)
> at org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:195)
> at org.apache.phoenix.schema.types.PDataType.checkForSufficientLength(PDataType.java:290)
> at org.apache.phoenix.schema.types.PLong$LongCodec.decodeLong(PLong.java:256)
> at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:115)
> at org.apache.phoenix.schema.types.PLong.toObject(PLong.java:31)
> at org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:1011)
> at org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:75)
> at org.apache.phoenix.jdbc.PhoenixResultSet.getObject(PhoenixResultSet.java:585)
> at sqlline.Rows$Row.<init>(Rows.java:258)
> at sqlline.BufferedRows.nextList(BufferedRows.java:111)
> at sqlline.BufferedRows.<init>(BufferedRows.java:52)
> at sqlline.SqlLine.print(SqlLine.java:1623)
> at sqlline.Commands.execute(Commands.java:982)
> at sqlline.Commands.sql(Commands.java:906)
> at sqlline.SqlLine.dispatch(SqlLine.java:740)
> at sqlline.SqlLine.begin(SqlLine.java:557)
> at sqlline.SqlLine.start(SqlLine.java:270)
> at sqlline.SqlLine.main(SqlLine.java:201)
> {code}
> I tried to drop the view, and I was able to query the data from the SYSCATA.
> I tested on 4.x-HBase1.3 and master branch, all branches have the same behavior.
>
> cc [~kadir] [~gjacoby] [~swaroopa]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)