You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Amarnath Ramamoorthi (JIRA)" <ji...@apache.org> on 2019/01/11 16:50:00 UTC
[jira] [Created] (PHOENIX-5097) Index Scrutiny Tool changes schema
name to UPPERCASE
Amarnath Ramamoorthi created PHOENIX-5097:
---------------------------------------------
Summary: Index Scrutiny Tool changes schema name to UPPERCASE
Key: PHOENIX-5097
URL: https://issues.apache.org/jira/browse/PHOENIX-5097
Project: Phoenix
Issue Type: Bug
Reporter: Amarnath Ramamoorthi
Creating index table and running *Index Scrutiny Tool.*
{code:java}
CREATE INDEX IF NOT EXISTS "IDX_CAP_DEMO_TABLE" ON "CAP".DEMO_TABLE ("id_x") INCLUDE ("id_y");
hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s CAP -dt DEMO_TABLE -it IDX_FOO_DEMO_TABLE -o
{code}
Works fine without error, since the actual schema name is in uppercase (CAP).
However on using lowercase schema name (nocap) it converts it to UPPERCASE.
{code:java}
CREATE INDEX IF NOT EXISTS "IDX_NOCAP_DEMO_TABLE" ON "nocap".DEMO_TABLE ("id_x") INCLUDE ("id_x");
[amar@locahost ~]$ hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s nocap -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
...........
...........
...........
19/01/11 13:45:09 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
19/01/11 13:45:11 ERROR index.IndexScrutinyTool: An exception occurred while performing the indexing job: IllegalArgumentException: IDX_NOCAP_DEMO_TABLE is not an index table for NOCAP.DEMO_TABLE at:
java.lang.IllegalArgumentException: IDX_NOCAP_DEMO_TABLE is not an index table for NOCAP.DEMO_TABLE
at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.run(IndexScrutinyTool.java:394)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.main(IndexScrutinyTool.java:518)
{code}
It changes the schema name to uppercase, so input the schema name with quotes to ignore the case like
{code:java}
hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s "nocap" -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
{code}
The same error follows when you use the above command with just quotes. On using the following command
{code:java}
[amar@locahost ~]$ hbase org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s \"\"nocap\"\" -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
...........
...........
...........
19/01/11 10:34:18 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Running scrutiny [schemaName="nocap", dataTable=DEMO_TABLE, indexTable=IDX_NOCAP_DEMO_TABLE, useSnapshot=false, timestamp=1547202800130, batchSize=1000, outputBasePath=null, outputFormat=TABLE, outputMaxRows=1000000]
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper: SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.DEMO_TABLE
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Upsert statement used for output table: UPSERT INTO PHOENIX_INDEX_SCRUTINY ("SOURCE_TABLE", "TARGET_TABLE", "SCRUTINY_EXECUTE_TIME", "SOURCE_ROW_PK_HASH", "SOURCE_TS", "TARGET_TS", "HAS_TARGET_ROW", "id_x","id_y","id_z" .... ) VALUES (?, ?, ?, ?, ?, ?)
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper: SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.IDX_NOCAP_DEMO_TABLE
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Upsert statement used for output table: UPSERT INTO PHOENIX_INDEX_SCRUTINY ("SOURCE_TABLE", "TARGET_TABLE", "SCRUTINY_EXECUTE_TIME", "SOURCE_ROW_PK_HASH", "SOURCE_TS", "TARGET_TS", "HAS_TARGET_ROW", "id_x","id_y","id_z" ..... ) VALUES (?, ?, ?, ?, ?, ?)
19/01/11 10:34:21 INFO index.IndexScrutinyTool: Running Index Scrutiny in Background - Submit async and exit
19/01/11 10:34:23 ERROR mapreduce.PhoenixInputFormat: Failed to get the query plan with error [ERROR 1012 (42M03): Table undefined. tableName=NOCAP.DEMO_TABLE]
19/01/11 10:34:23 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/amar/.staging/job_1540390314309_0119
19/01/11 10:34:23 ERROR index.IndexScrutinyTool: An exception occurred while performing the indexing job: RuntimeException: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=NOCAP.DEMO_TABLE at:
java.lang.RuntimeException: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=NOCAP.DEMO_TABLE
at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:139)
at org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:81)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.run(IndexScrutinyTool.java:440)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.phoenix.mapreduce.index.IndexScrutinyTool.main(IndexScrutinyTool.java:518)
Caused by: org.apache.phoenix.schema.TableNotFoundException: ERROR 1012 (42M03): Table undefined. tableName=NOCAP.DEMO_TABLE
at org.apache.phoenix.compile.FromCompiler$BaseColumnResolver.createTableRef(FromCompiler.java:577)
at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.<init>(FromCompiler.java:391)
at org.apache.phoenix.compile.FromCompiler.getResolverForQuery(FromCompiler.java:228)
at org.apache.phoenix.compile.FromCompiler.getResolverForQuery(FromCompiler.java:206)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:468)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:442)
at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1679)
at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1672)
at org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1666)
at org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:120)
... 14 more
{code}
As you can see it reads as lowercase, if you look at the second line form the above output, which is
{code:java}
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Running scrutiny [schemaName="nocap", dataTable=DEMO_TABLE, indexTable=IDX_NOCAP_DEMO_TABLE, useSnapshot=false, timestamp=1547202800130, batchSize=1000, outputBasePath=null, outputFormat=TABLE, outputMaxRows=1000000]
{code}
and the very next line
{code:java}
19/01/11 10:34:20 INFO index.IndexScrutinyTool: Query used on source table to feed the mapper: SELECT /*+ NO_INDEX */ "id_x","id_y","id_z" FROM NOCAP.DEMO_TABLE
{code}
it changes the schema name to uppercase.
Tried other phoenix suggested option to run this "Index Scrutiny Tool" from Hadoop using phoenix-server.jar
{code:java}
HADOOP_CLASSPATH=$(hbase mapredcp) hadoop jar phoenix4.13-cdh5.9.2-marin-1.5.1-server.jar org.apache.phoenix.mapreduce.index.IndexScrutinyTool -s \"\"nocap\"\" -dt DEMO_TABLE -it IDX_NOCAP_DEMO_TABLE -o
{code}
However, I believe this ends up with the same error, since using the same module "org.apache.phoenix.mapreduce.index.IndexScrutinyTool"
Hope there is enough information here.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)