You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by "Pindi, Srinivas" <sr...@epsilon.com> on 2016/04/13 20:47:27 UTC
Undefined column. columnName=IS_ROW_TIMESTAMP
Problem Statement:
While we are trying to create a phoenix view and we are getting the following exception.
When I looked at the source code here is what I found:
if (currentServerSideTableTimeStamp < MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP_4_6_0) {
columnsToAdd += PhoenixDatabaseMetaData.IS_ROW_TIMESTAMP + " " + PBoolean.INSTANCE.getSqlTypeName();
}
Version:
Server phoenix-4.6.1-HBase-1.0-1.17.0.21
Client phoenix-4.6.1-HBase-1.0-1.17.0.21
Error:
Caused by: java.sql.SQLException: Error running CREATE VIEW XYZ ("A" VARCHAR, "B" VARCHAR) AS SELECT * FROM HBASE_TABLE_1 WHERE (KEY >= '0-02UsG-0' AND KEY <= '0-02UsG-g') AND UPDATED_DATE IS NOT NULL
...
...
... 90 more
Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined column. columnName=IS_ROW_TIMESTAMP
at org.apache.phoenix.schema.PTableImpl.getColumn(PTableImpl.java:647)
at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:361)
at org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:386)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:553)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:541)
at org.apache.phoenix.jdbc.PhoenixPreparedStatement.getParameterMetaData(PhoenixPreparedStatement.java:246)
at org.apache.phoenix.schema.MetaDataClient.addColumnMutation(MetaDataClient.java:707)
at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1943)
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Ankit,
We have resolved this issue by adding HBase puts for IS_ROW_TIMESTAMP, and BASE_COLUMN_COUNTS at timestamps 9, and 8 respectively.
My understanding is that this issue was occurring when phoenix scan results spills to disk, which contains HBase scan results. Since these two columns are not showing in HBase scan its throwing error as “Undefined column”.
We are not sure why Phoenix upgrade did not add these two columns in HBase though.
Thank you very much for your time and suggestions,
Bharathi.
From: Bavineni, Bharata
Sent: Monday, May 02, 2016 9:33 AM
To: user@phoenix.apache.org
Subject: RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Yes. I tried restarting cluster as well with these steps.
I will send debug results.
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Monday, May 02, 2016 4:29 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Have you restarted the cluster to refresh the cache? If the rows got appear again in system.stats then delete the rows again, restart the cluster and try altering the table.
If this doesn't solves the problem, I think , you should try attaching debugger and share your findings.
On Mon, May 2, 2016 at 6:20 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
I deleted SYSTEM.CATALOG entry from SYSTEM.STATS, but still getting same error for Alter statement.
I have also tried these below steps:
- Enable KEEP_DELETED_CELLS to false for SYSTEM.STATS
- flush ‘SYSTEM.STATS’
- major_compact ‘SYSTEM.STATS’
Still I do see one row from hbase raw scan. Please find attached the results of this scan:
scan 'SYSTEM.STATS',{ RAW => true, STARTROW => "SYSTEM.CATALOG\x000", STOPROW => "SYSTEM.CATALOG\x001"}
Could this row be causing the issue?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 1:36 PM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
You have guidePosts collected for SYSTEM.CATALOG table which is actually causing the problem. It is possible only if your guidePosts width was set anytime low or you have very big system.catalog table.
can you please delete guideposts from system.catalog table(delete from SYSTEM.STATS where physical_name='SYSTEM.CATALOG') and try adding a column.
On Sun, May 1, 2016 at 9:02 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
Please find attached results for steps 1, and 3.
SYSTEM.SEQUENCE is returning results for all the tables. Are you looking for any specific data? so that I can filter and send the results?
Thank you for your time looking into this,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 1:01 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
It's weird that the exception is still there.
is it possible for you to share following outputs.
1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
2. scan 'SYSTEM.SEQUENCE',{RAW => true}
3. from phoenix , select * from system.stats where PHYSICAL_NAME='SYSTEM.CATALOG';
On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Makes sense.
Restarting HBase cluster did not seem to fix the issue. Still getting same error for Alter command:
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
Any other suggestions?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:44 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Explicit "put" to add column is not recommended as it may affect you down the line when you start using features which expects some column attribute and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Yes. I tried restarting cluster as well with these steps.
I will send debug results.
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Monday, May 02, 2016 4:29 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Have you restarted the cluster to refresh the cache? If the rows got appear again in system.stats then delete the rows again, restart the cluster and try altering the table.
If this doesn't solves the problem, I think , you should try attaching debugger and share your findings.
On Mon, May 2, 2016 at 6:20 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
I deleted SYSTEM.CATALOG entry from SYSTEM.STATS, but still getting same error for Alter statement.
I have also tried these below steps:
- Enable KEEP_DELETED_CELLS to false for SYSTEM.STATS
- flush ‘SYSTEM.STATS’
- major_compact ‘SYSTEM.STATS’
Still I do see one row from hbase raw scan. Please find attached the results of this scan:
scan 'SYSTEM.STATS',{ RAW => true, STARTROW => "SYSTEM.CATALOG\x000", STOPROW => "SYSTEM.CATALOG\x001"}
Could this row be causing the issue?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 1:36 PM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
You have guidePosts collected for SYSTEM.CATALOG table which is actually causing the problem. It is possible only if your guidePosts width was set anytime low or you have very big system.catalog table.
can you please delete guideposts from system.catalog table(delete from SYSTEM.STATS where physical_name='SYSTEM.CATALOG') and try adding a column.
On Sun, May 1, 2016 at 9:02 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
Please find attached results for steps 1, and 3.
SYSTEM.SEQUENCE is returning results for all the tables. Are you looking for any specific data? so that I can filter and send the results?
Thank you for your time looking into this,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 1:01 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
It's weird that the exception is still there.
is it possible for you to share following outputs.
1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
2. scan 'SYSTEM.SEQUENCE',{RAW => true}
3. from phoenix , select * from system.stats where PHYSICAL_NAME='SYSTEM.CATALOG';
On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Makes sense.
Restarting HBase cluster did not seem to fix the issue. Still getting same error for Alter command:
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
Any other suggestions?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:44 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Explicit "put" to add column is not recommended as it may affect you down the line when you start using features which expects some column attribute and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Have you restarted the cluster to refresh the cache? If the rows got appear
again in system.stats then delete the rows again, restart the cluster and
try altering the table.
If this doesn't solves the problem, I think , you should try attaching
debugger and share your findings.
On Mon, May 2, 2016 at 6:20 AM, Bavineni, Bharata <
Bharata.Bavineni@epsilon.com> wrote:
> Ankit,
>
> I deleted SYSTEM.CATALOG entry from SYSTEM.STATS, but still getting same
> error for Alter statement.
>
> I have also tried these below steps:
>
> - Enable KEEP_DELETED_CELLS to false for SYSTEM.STATS
>
> - flush ‘SYSTEM.STATS’
>
> - major_compact ‘SYSTEM.STATS’
>
>
>
> Still I do see one row from hbase raw scan. Please find attached the
> results of this scan:
>
> scan 'SYSTEM.STATS',{ RAW => true, STARTROW => "SYSTEM.CATALOG\x000",
> STOPROW => "SYSTEM.CATALOG\x001"}
>
> Could this row be causing the issue?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 1:36 PM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> You have guidePosts collected for SYSTEM.CATALOG table which is actually
> causing the problem. It is possible only if your guidePosts width was set
> anytime low or you have very big system.catalog table.
>
>
>
> can you please delete guideposts from system.catalog table(delete from
> SYSTEM.STATS where physical_name='SYSTEM.CATALOG') and try adding a column.
>
>
>
> On Sun, May 1, 2016 at 9:02 PM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Hi Ankit,
>
> Please find attached results for steps 1, and 3.
>
> SYSTEM.SEQUENCE is returning results for all the tables. Are you looking
> for any specific data? so that I can filter and send the results?
>
>
>
> Thank you for your time looking into this,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 1:01 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> It's weird that the exception is still there.
>
> is it possible for you to share following outputs.
>
>
>
> 1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
>
> 2. scan 'SYSTEM.SEQUENCE',{RAW => true}
>
> 3. from phoenix , select * from system.stats
> where PHYSICAL_NAME='SYSTEM.CATALOG';
>
>
>
>
>
> On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Makes sense.
>
> Restarting HBase cluster did not seem to fix the issue. Still getting same
> error for Alter command:
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
>
>
> Any other suggestions?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:44 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Explicit "put" to add column is not recommended as it may affect you down
> the line when you start using features which expects some column attribute
> and are missed because of manual error.
>
> It could be just a solution for immediate production recovery.
>
>
>
>
>
> On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Ankit,
>
> We will try restarting HBase cluster. Adding explicit “put” in HBase with
> timestamp as 9 for these two columns has any side effects?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:35 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> As you are also not seeing any cell having timestamp greater than 9 (in scan
> 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"})
> then I think restarting your hbase cluster will help as it will bounce the
> stale cache.
>
> Nothing is required to be deleted from phoenix side.
>
>
>
> On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Ankit,
I deleted SYSTEM.CATALOG entry from SYSTEM.STATS, but still getting same error for Alter statement.
I have also tried these below steps:
- Enable KEEP_DELETED_CELLS to false for SYSTEM.STATS
- flush ‘SYSTEM.STATS’
- major_compact ‘SYSTEM.STATS’
Still I do see one row from hbase raw scan. Please find attached the results of this scan:
scan 'SYSTEM.STATS',{ RAW => true, STARTROW => "SYSTEM.CATALOG\x000", STOPROW => "SYSTEM.CATALOG\x001"}
Could this row be causing the issue?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Sunday, May 01, 2016 1:36 PM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
You have guidePosts collected for SYSTEM.CATALOG table which is actually causing the problem. It is possible only if your guidePosts width was set anytime low or you have very big system.catalog table.
can you please delete guideposts from system.catalog table(delete from SYSTEM.STATS where physical_name='SYSTEM.CATALOG') and try adding a column.
On Sun, May 1, 2016 at 9:02 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
Please find attached results for steps 1, and 3.
SYSTEM.SEQUENCE is returning results for all the tables. Are you looking for any specific data? so that I can filter and send the results?
Thank you for your time looking into this,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 1:01 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
It's weird that the exception is still there.
is it possible for you to share following outputs.
1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
2. scan 'SYSTEM.SEQUENCE',{RAW => true}
3. from phoenix , select * from system.stats where PHYSICAL_NAME='SYSTEM.CATALOG';
On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Makes sense.
Restarting HBase cluster did not seem to fix the issue. Still getting same error for Alter command:
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
Any other suggestions?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:44 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Explicit "put" to add column is not recommended as it may affect you down the line when you start using features which expects some column attribute and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
You have guidePosts collected for SYSTEM.CATALOG table which is actually
causing the problem. It is possible only if your guidePosts width was set
anytime low or you have very big system.catalog table.
can you please delete guideposts from system.catalog table(delete from
SYSTEM.STATS where physical_name='SYSTEM.CATALOG') and try adding a column.
On Sun, May 1, 2016 at 9:02 PM, Bavineni, Bharata <
Bharata.Bavineni@epsilon.com> wrote:
> Hi Ankit,
>
> Please find attached results for steps 1, and 3.
>
> SYSTEM.SEQUENCE is returning results for all the tables. Are you looking
> for any specific data? so that I can filter and send the results?
>
>
>
> Thank you for your time looking into this,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 1:01 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> It's weird that the exception is still there.
>
> is it possible for you to share following outputs.
>
>
>
> 1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
>
> 2. scan 'SYSTEM.SEQUENCE',{RAW => true}
>
> 3. from phoenix , select * from system.stats
> where PHYSICAL_NAME='SYSTEM.CATALOG';
>
>
>
>
>
> On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Makes sense.
>
> Restarting HBase cluster did not seem to fix the issue. Still getting same
> error for Alter command:
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
>
>
> Any other suggestions?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:44 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Explicit "put" to add column is not recommended as it may affect you down
> the line when you start using features which expects some column attribute
> and are missed because of manual error.
>
> It could be just a solution for immediate production recovery.
>
>
>
>
>
> On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Ankit,
>
> We will try restarting HBase cluster. Adding explicit “put” in HBase with
> timestamp as 9 for these two columns has any side effects?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:35 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> As you are also not seeing any cell having timestamp greater than 9 (in scan
> 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"})
> then I think restarting your hbase cluster will help as it will bounce the
> stale cache.
>
> Nothing is required to be deleted from phoenix side.
>
>
>
> On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Hi Ankit,
Please find attached results for steps 1, and 3.
SYSTEM.SEQUENCE is returning results for all the tables. Are you looking for any specific data? so that I can filter and send the results?
Thank you for your time looking into this,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Sunday, May 01, 2016 1:01 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
It's weird that the exception is still there.
is it possible for you to share following outputs.
1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
2. scan 'SYSTEM.SEQUENCE',{RAW => true}
3. from phoenix , select * from system.stats where PHYSICAL_NAME='SYSTEM.CATALOG';
On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Makes sense.
Restarting HBase cluster did not seem to fix the issue. Still getting same error for Alter command:
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
Any other suggestions?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:44 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Explicit "put" to add column is not recommended as it may affect you down the line when you start using features which expects some column attribute and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
It's weird that the exception is still there.
is it possible for you to share following outputs.
1. scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\
x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}
2. scan 'SYSTEM.SEQUENCE',{RAW => true}
3. from phoenix , select * from system.stats
where PHYSICAL_NAME='SYSTEM.CATALOG';
On Sun, May 1, 2016 at 11:19 AM, Bavineni, Bharata <
Bharata.Bavineni@epsilon.com> wrote:
> Makes sense.
>
> Restarting HBase cluster did not seem to fix the issue. Still getting same
> error for Alter command:
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
>
>
> Any other suggestions?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:44 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Explicit "put" to add column is not recommended as it may affect you down
> the line when you start using features which expects some column attribute
> and are missed because of manual error.
>
> It could be just a solution for immediate production recovery.
>
>
>
>
>
> On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Ankit,
>
> We will try restarting HBase cluster. Adding explicit “put” in HBase with
> timestamp as 9 for these two columns has any side effects?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:35 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> As you are also not seeing any cell having timestamp greater than 9 (in scan
> 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"})
> then I think restarting your hbase cluster will help as it will bounce the
> stale cache.
>
> Nothing is required to be deleted from phoenix side.
>
>
>
> On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Makes sense.
Restarting HBase cluster did not seem to fix the issue. Still getting same error for Alter command:
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
Any other suggestions?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Sunday, May 01, 2016 12:44 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Explicit "put" to add column is not recommended as it may affect you down the line when you start using features which expects some column attribute and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Explicit "put" to add column is not recommended as it may affect you down
the line when you start using features which expects some column attribute
and are missed because of manual error.
It could be just a solution for immediate production recovery.
On Sun, May 1, 2016 at 11:07 AM, Bavineni, Bharata <
Bharata.Bavineni@epsilon.com> wrote:
> Ankit,
>
> We will try restarting HBase cluster. Adding explicit “put” in HBase with
> timestamp as 9 for these two columns has any side effects?
>
>
>
> Thank you,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Sunday, May 01, 2016 12:35 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> As you are also not seeing any cell having timestamp greater than 9 (in scan
> 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"})
> then I think restarting your hbase cluster will help as it will bounce the
> stale cache.
>
> Nothing is required to be deleted from phoenix side.
>
>
>
> On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
> Bharata.Bavineni@epsilon.com> wrote:
>
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Ankit,
We will try restarting HBase cluster. Adding explicit “put” in HBase with timestamp as 9 for these two columns has any side effects?
Thank you,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Sunday, May 01, 2016 12:35 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
As you are also not seeing any cell having timestamp greater than 9 (in scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <Bh...@epsilon.com>> wrote:
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com<ma...@gmail.com>]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org<ma...@phoenix.apache.org>
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
As you are also not seeing any cell having timestamp greater than 9 (in scan
'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00", STOPROW
=> "\x00SYSTEM\x00CATALOG\x01"}) then I think restarting your hbase cluster
will help as it will bounce the stale cache.
Nothing is required to be deleted from phoenix side.
On Sat, Apr 30, 2016 at 9:04 PM, Bavineni, Bharata <
Bharata.Bavineni@epsilon.com> wrote:
> Hi Ankit,
>
> We have removed all cells for these two columns and scan with raw =>
> true does not return any results:
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
>
> scan 'SYSTEM.CATALOG',{RAW => true, STARTROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW =>
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
>
> We still see the same error.
>
>
>
> Do we need to explicitly delete from phoenix as well?
>
>
>
> Thanks,
>
> Bharathi.
>
>
>
> *From:* Ankit Singhal [mailto:ankitsinghal59@gmail.com]
> *Sent:* Saturday, April 30, 2016 3:00 AM
> *To:* user@phoenix.apache.org
> *Subject:* Re: Undefined column. columnName=IS_ROW_TIMESTAMP
>
>
>
> Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
> should not be any cell for SYSTEM.CATALOG row having timestamp greater than
> 9.
>
>
>
> On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
>
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
>
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
>
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>
> at sqlline.Commands.execute(Commands.java:822)
>
> at sqlline.Commands.sql(Commands.java:732)
>
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>
> at sqlline.SqlLine.begin(SqlLine.java:681)
>
> at sqlline.SqlLine.start(SqlLine.java:398)
>
> at sqlline.SqlLine.main(SqlLine.java:292)
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
>
>
> bin/sqlline.py "localhost;currentSCN=9"
>
>
>
> So, can you fix the timestamp for these columns?
>
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
>
> - Take snapshot of your system.catalog( for backup)
>
> - delete such cells from system.catalog at latest timestamp
>
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
>
> - do flush and major_compaction on SYSTEM.CATALOG
>
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
>
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Arun,
>
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
>
> Thanks,
>
> James
>
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> Hi Ankit,
>
>
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
>
> Will this cause an issue in the future if it has to compare the timestamps?
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Hi Ankit,
>
>
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
>
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
>
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
>
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Yes Arun, it seems fine to me.
>
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
>
>
> And, can you also please share output of
>
> > scan 'SYSTEM.SEQUENCE'
>
>
>
> Regards,
>
> Ankit
>
>
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
>
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,
>
>
>
> When tried executing the query, the following exception is thrown.
>
>
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
>
> Thank,
> Arun
>
>
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
>
>
> But anyways, let's add the required columns manually by following below
> step;
>
>
>
> > ./sqlline.py localhost;CurrentSCN=9
>
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
>
> >!quit
>
>
>
> Quit the shell and start new session without CurrentSCN.
>
> > ./sqlline.py localhost
>
> > !describe system.catalog
>
>
>
> this should resolve the issue of missing column.
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Ankit,James,
>
>
>
> Please find the results you have asked for in the attachment text document.
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Arun,
>
> the observations you had is understandable.
>
> So, that's why we again requested the output of
>
>
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
>
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
>
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
>
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
>
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
>
>
> Is this an expected behavior?
>
>
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
>
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> One more question to add,
>
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
>
>
> I have limited the scan in HBase and here is the following data.
>
>
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> Table SYSTEM.CATALOG is ENABLED
>
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
>
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
>
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
>
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
>
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>
> COLUMN FAMILIES DESCRIPTION
>
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
>
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
>
>
> 1 row in 0.6060 seconds
>
>
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
> Arun,
>
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
>
> Thanks,
>
> James
>
>
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
> Hi Arun,
>
>
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
>
>
>
> if not,
>
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
>
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
>
>
>
>
> Regards,
>
> Ankit Singhal
>
>
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
>
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
>
>
>
> Thanks,
>
> Arun
>
>
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
> James,
>
>
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
>
>
> Thanks,
>
> Arun
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
RE: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Bavineni, Bharata" <Bh...@epsilon.com>.
Hi Ankit,
We have removed all cells for these two columns and scan with raw => true does not return any results:
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00BASE_COLUMN_COUNT_\x00"}
scan 'SYSTEM.CATALOG',{RAW => true, STARTROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x00", STOPROW => "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP_\x00"}
We still see the same error.
Do we need to explicitly delete from phoenix as well?
Thanks,
Bharathi.
From: Ankit Singhal [mailto:ankitsinghal59@gmail.com]
Sent: Saturday, April 30, 2016 3:00 AM
To: user@phoenix.apache.org
Subject: Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there should not be any cell for SYSTEM.CATALOG row having timestamp greater than 9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG (state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";" as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>> wrote:
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Just following with the question that when the alter statement was issued with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current timestamp was not set to 9. Will this cause an issue in the future if it has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the following is the output.
hbase(main):004:0> get 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655, value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655, value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655, value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655, value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>> wrote:
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work correctly.(Remember use correct INTEGER byte representation for DATA_TYPE column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG', "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>> wrote:
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG
So that we can help you in adding ROW_TIMESTAMP column in system.catalog table so that your phoenix setup works properly and you can also see it in !describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do? This issue is blocking us in the production environment. Any help to resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>> wrote:
It's ok if you can just post after grep for CATALOG in a command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY, timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE, timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655, value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655, value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655, value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655, value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655, value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 => '|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 => '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|', coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 => '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA => {'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan 'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>> wrote:
Arun,
Please run the command Ankit mentioned in an HBase shell and post the output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>> wrote:
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and which stopping upgrade code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
After further investigation, we found that Phoenix Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <ar...@gmail.com>> wrote:
James,
To add more information on this issue, this happens in new phoenix views associated with brand new tables as well. So, this cannot be an upgrade/migration issue. Not figured out a specific way to reproduce this issue yet. Could you throw some ideas on what direction this problem could be approached from this point?
Thanks,
Arun
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Do a raw scan on SYSTEM.CATALOG through hbase shell and confirm that there
should not be any cell for SYSTEM.CATALOG row having timestamp greater than
9.
On Sat, Apr 30, 2016 at 3:28 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Ankit,
>
> Follwing the steps, the below exception is thrown. Can you let me know
> what could have gone wrong?
>
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
> BOOLEAN;
> Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
> (state=42M04,code=1013)
> org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
> (42M04): Table already exists. tableName=SYSTEM.CATALOG
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:292)
>
> Thanks,
> Arun
>
> On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
>> Hi Arun,
>>
>> Actually , we missed the quotes as shell will consider everything after
>> ";" as second command.
>>
>> bin/sqlline.py "localhost;currentSCN=9"
>>
>> So, can you fix the timestamp for these columns?
>> Steps you may follow(but request you to try first on non production
>> cluster as these are not tested by me).
>> - Take snapshot of your system.catalog( for backup)
>> - delete such cells from system.catalog at latest timestamp
>> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
>> not have any query running and using schema in point in time snapshots).
>> - do flush and major_compaction on SYSTEM.CATALOG
>> - when you don't see those columns and open connection at currentSCN=9
>> and alter table to add both the columns.
>> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>>
>> Regards,
>> Ankit Singhal
>>
>>
>>
>> Regards,
>> Ankit Singhal
>>
>> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> James,
>>>
>>> Thanks for the answer. But, why was the currentSCN=9 on connection is
>>> not working in the first place? Do you have any pointers for us to look
>>> into?
>>>
>>> Thanks,
>>> Arun
>>>
>>> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
>>> wrote:
>>>
>>>> Hi Arun,
>>>> Yes, this could cause problems, as we base the need to upgrade on the
>>>> time stamp of the system catalog table.
>>>> Thanks,
>>>> James
>>>>
>>>>
>>>> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Hi Ankit,
>>>>>
>>>>> Just following with the question that when the alter statement was
>>>>> issued with CurrentSCN=9, the current timestamp was not set to 9.
>>>>> Will this cause an issue in the future if it has to compare the
>>>>> timestamps?
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> Hi Ankit,
>>>>>>
>>>>>> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT
>>>>>> INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it
>>>>>> actually added the columns to the SYSTEM.CATALOG.
>>>>>> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The
>>>>>> current timestamp was not set to 9. Will this cause an issue in the future
>>>>>> if it has to compare the timestamps?
>>>>>>
>>>>>> After the alter statement was executed and when queried from HBase,
>>>>>> the following is the output.
>>>>>>
>>>>>> hbase(main):004:0> get
>>>>>> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
>>>>>> COLUMN CELL
>>>>>> 0 ATA_TYPE timestamp=1461362240655,
>>>>>> value=\x80\x00\x00\x10
>>>>>> 0:IS_VIEW_REFERENCED timestamp=1461362240655,
>>>>>> value=\x00
>>>>>> 0:NULLABLE timestamp=1461362240655,
>>>>>> value=\x80\x00\x00\x01
>>>>>> 0 RDINAL_POSITION timestamp=1461362240655,
>>>>>> value=\x80\x00\x004
>>>>>> 0 ORT_ORDER timestamp=1461362240655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <
>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>
>>>>>>> Yes Arun, it seems fine to me.
>>>>>>> Can you do it for BASE_COLUMN_COUNT too so that the dependent
>>>>>>> features work correctly.(Remember use correct INTEGER byte representation
>>>>>>> for DATA_TYPE column).
>>>>>>>
>>>>>>> And, can you also please share output of
>>>>>>> > scan 'SYSTEM.SEQUENCE'
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ankit
>>>>>>>
>>>>>>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> Ankit,
>>>>>>>>
>>>>>>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did
>>>>>>>> the job.
>>>>>>>> Is it okay to do the following? These puts in HBase intends to do
>>>>>>>> what the ALTER command in phoenix supposed to do.
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>>>>>>
>>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Arun
>>>>>>>>
>>>>>>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Ankit,
>>>>>>>>>
>>>>>>>>> When tried executing the query, the following exception is thrown.
>>>>>>>>>
>>>>>>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>>>>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>>>>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR
>>>>>>>>> 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>>>>>>
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>>>>>>> at
>>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>>>>>>> at sqlline.Commands.execute(Commands.java:822)
>>>>>>>>> at sqlline.Commands.sql(Commands.java:732)
>>>>>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>>>>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>>>>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Arun,
>>>>>>>>>> As per the system.catalog ,there seems no way that upgrade
>>>>>>>>>> code(adding new columns and view upgrade for version < v4.5) doesn't
>>>>>>>>>> execute.
>>>>>>>>>>
>>>>>>>>>> But anyways, let's add the required columns manually by following
>>>>>>>>>> below step;
>>>>>>>>>>
>>>>>>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>>>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>>>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>>>>>>> >!quit
>>>>>>>>>>
>>>>>>>>>> Quit the shell and start new session without CurrentSCN.
>>>>>>>>>> > ./sqlline.py localhost
>>>>>>>>>> > !describe system.catalog
>>>>>>>>>>
>>>>>>>>>> this should resolve the issue of missing column.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ankit Singhal
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Ankit,James,
>>>>>>>>>>>
>>>>>>>>>>> Please find the results you have asked for in the attachment
>>>>>>>>>>> text document.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Arun,
>>>>>>>>>>>> the observations you had is understandable.
>>>>>>>>>>>> So, that's why we again requested the output of
>>>>>>>>>>>>
>>>>>>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>>>>>>> CATALOG*
>>>>>>>>>>>>
>>>>>>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>>>>>>> also see it in !describe command.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>>>>>>
>>>>>>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix
>>>>>>>>>>>>> 4.6.1 client,
>>>>>>>>>>>>>
>>>>>>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>>>>>>
>>>>>>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP
>>>>>>>>>>>>> column.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this an expected behavior?
>>>>>>>>>>>>>
>>>>>>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what
>>>>>>>>>>>>> should I do? This issue is blocking us in the production environment. Any
>>>>>>>>>>>>> help to resolve or workaround is highly appreciated.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> It's ok if you can just post after grep for CATALOG in a
>>>>>>>>>>>>>> command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> One more question to add,
>>>>>>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have limited the scan in HBase and here is the following
>>>>>>>>>>>>>>> data.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>>>>>>> '|org.apache
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' =>
>>>>>>>>>>>>>>>> 'true'}}
>>>>>>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF',
>>>>>>>>>>>>>>>> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE',
>>>>>>>>>>>>>>>> VERSIONS => '1000',
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS
>>>>>>>>>>>>>>>> => 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Arun,
>>>>>>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell
>>>>>>>>>>>>>>>>> and post the output back here.
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> James
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning
>>>>>>>>>>>>>>>>>> IS_ROW_TIMESTAMP column.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> select * from SYSTEM.CATALOG where
>>>>>>>>>>>>>>>>>> TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG,
>>>>>>>>>>>>>>>>>>> by doing !describe on system.catalog.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran
>>>>>>>>>>>>>>>>>>> Sabtharishi <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> After further investigation, we found that Phoenix
>>>>>>>>>>>>>>>>>>>> Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7,
>>>>>>>>>>>>>>>>>>>> and 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran
>>>>>>>>>>>>>>>>>>>> Sabtharishi <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To add more information on this issue, this happens in
>>>>>>>>>>>>>>>>>>>>> new phoenix views associated with brand new tables as well. So, this cannot
>>>>>>>>>>>>>>>>>>>>> be an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Ankit,
Follwing the steps, the below exception is thrown. Can you let me know what
could have gone wrong?
ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER, IS_ROW_TIMESTAMP
BOOLEAN;
Error: ERROR 1013 (42M04): Table already exists. tableName=SYSTEM.CATALOG
(state=42M04,code=1013)
org.apache.phoenix.schema.NewerTableAlreadyExistsException: ERROR 1013
(42M04): Table already exists. tableName=SYSTEM.CATALOG
at
org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2345)
at
org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
at sqlline.SqlLine.main(SqlLine.java:292)
Thanks,
Arun
On Fri, Apr 29, 2016 at 4:47 AM, Ankit Singhal <an...@gmail.com>
wrote:
> Hi Arun,
>
> Actually , we missed the quotes as shell will consider everything after
> ";" as second command.
>
> bin/sqlline.py "localhost;currentSCN=9"
>
> So, can you fix the timestamp for these columns?
> Steps you may follow(but request you to try first on non production
> cluster as these are not tested by me).
> - Take snapshot of your system.catalog( for backup)
> - delete such cells from system.catalog at latest timestamp
> - set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should
> not have any query running and using schema in point in time snapshots).
> - do flush and major_compaction on SYSTEM.CATALOG
> - when you don't see those columns and open connection at currentSCN=9 and
> alter table to add both the columns.
> - you may set keep_deleted_cells back to true in SYSTEM.CATALOG
>
> Regards,
> Ankit Singhal
>
>
>
> Regards,
> Ankit Singhal
>
> On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> James,
>>
>> Thanks for the answer. But, why was the currentSCN=9 on connection is not
>> working in the first place? Do you have any pointers for us to look into?
>>
>> Thanks,
>> Arun
>>
>> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
>> wrote:
>>
>>> Hi Arun,
>>> Yes, this could cause problems, as we base the need to upgrade on the
>>> time stamp of the system catalog table.
>>> Thanks,
>>> James
>>>
>>>
>>> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
>>> wrote:
>>>
>>>> Hi Ankit,
>>>>
>>>> Just following with the question that when the alter statement was
>>>> issued with CurrentSCN=9, the current timestamp was not set to 9.
>>>> Will this cause an issue in the future if it has to compare the
>>>> timestamps?
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Hi Ankit,
>>>>>
>>>>> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT
>>>>> INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it
>>>>> actually added the columns to the SYSTEM.CATALOG.
>>>>> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The
>>>>> current timestamp was not set to 9. Will this cause an issue in the future
>>>>> if it has to compare the timestamps?
>>>>>
>>>>> After the alter statement was executed and when queried from HBase,
>>>>> the following is the output.
>>>>>
>>>>> hbase(main):004:0> get
>>>>> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
>>>>> COLUMN CELL
>>>>> 0 ATA_TYPE timestamp=1461362240655,
>>>>> value=\x80\x00\x00\x10
>>>>> 0:IS_VIEW_REFERENCED timestamp=1461362240655,
>>>>> value=\x00
>>>>> 0:NULLABLE timestamp=1461362240655,
>>>>> value=\x80\x00\x00\x01
>>>>> 0 RDINAL_POSITION timestamp=1461362240655,
>>>>> value=\x80\x00\x004
>>>>> 0 ORT_ORDER timestamp=1461362240655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>>
>>>>> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>
>>>>>> Yes Arun, it seems fine to me.
>>>>>> Can you do it for BASE_COLUMN_COUNT too so that the dependent
>>>>>> features work correctly.(Remember use correct INTEGER byte representation
>>>>>> for DATA_TYPE column).
>>>>>>
>>>>>> And, can you also please share output of
>>>>>> > scan 'SYSTEM.SEQUENCE'
>>>>>>
>>>>>> Regards,
>>>>>> Ankit
>>>>>>
>>>>>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> Ankit,
>>>>>>>
>>>>>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did
>>>>>>> the job.
>>>>>>> Is it okay to do the following? These puts in HBase intends to do
>>>>>>> what the ALTER command in phoenix supposed to do.
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>>>>>
>>>>>>> put 'SYSTEM.CATALOG',
>>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Arun
>>>>>>>
>>>>>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> Ankit,
>>>>>>>>
>>>>>>>> When tried executing the query, the following exception is thrown.
>>>>>>>>
>>>>>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>>>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>>>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR
>>>>>>>> 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>>>>>
>>>>>>>> at
>>>>>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>>>>>> at
>>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>>>>>> at sqlline.Commands.execute(Commands.java:822)
>>>>>>>> at sqlline.Commands.sql(Commands.java:732)
>>>>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>>>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>>>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>>>>>
>>>>>>>>
>>>>>>>> Thank,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Arun,
>>>>>>>>> As per the system.catalog ,there seems no way that upgrade
>>>>>>>>> code(adding new columns and view upgrade for version < v4.5) doesn't
>>>>>>>>> execute.
>>>>>>>>>
>>>>>>>>> But anyways, let's add the required columns manually by following
>>>>>>>>> below step;
>>>>>>>>>
>>>>>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>>>>>> >!quit
>>>>>>>>>
>>>>>>>>> Quit the shell and start new session without CurrentSCN.
>>>>>>>>> > ./sqlline.py localhost
>>>>>>>>> > !describe system.catalog
>>>>>>>>>
>>>>>>>>> this should resolve the issue of missing column.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ankit Singhal
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Ankit,James,
>>>>>>>>>>
>>>>>>>>>> Please find the results you have asked for in the attachment text
>>>>>>>>>> document.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Arun,
>>>>>>>>>>> the observations you had is understandable.
>>>>>>>>>>> So, that's why we again requested the output of
>>>>>>>>>>>
>>>>>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>>>>>> CATALOG*
>>>>>>>>>>>
>>>>>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>>>>>> also see it in !describe command.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>>>>>
>>>>>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix
>>>>>>>>>>>> 4.6.1 client,
>>>>>>>>>>>>
>>>>>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>>>>>
>>>>>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP
>>>>>>>>>>>> column.
>>>>>>>>>>>>
>>>>>>>>>>>> Is this an expected behavior?
>>>>>>>>>>>>
>>>>>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what
>>>>>>>>>>>> should I do? This issue is blocking us in the production environment. Any
>>>>>>>>>>>> help to resolve or workaround is highly appreciated.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> It's ok if you can just post after grep for CATALOG in a
>>>>>>>>>>>>> command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> One more question to add,
>>>>>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have limited the scan in HBase and here is the following
>>>>>>>>>>>>>> data.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>>>>>> '|org.apache
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' =>
>>>>>>>>>>>>>>> 'true'}}
>>>>>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF',
>>>>>>>>>>>>>>> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE',
>>>>>>>>>>>>>>> VERSIONS => '1000',
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Arun,
>>>>>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell
>>>>>>>>>>>>>>>> and post the output back here.
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> James
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>>>>>>> column.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> select * from SYSTEM.CATALOG where
>>>>>>>>>>>>>>>>> TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG,
>>>>>>>>>>>>>>>>>> by doing !describe on system.catalog.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi
>>>>>>>>>>>>>>>>>> <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After further investigation, we found that Phoenix
>>>>>>>>>>>>>>>>>>> Upsert query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and
>>>>>>>>>>>>>>>>>>> 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran
>>>>>>>>>>>>>>>>>>> Sabtharishi <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> To add more information on this issue, this happens in
>>>>>>>>>>>>>>>>>>>> new phoenix views associated with brand new tables as well. So, this cannot
>>>>>>>>>>>>>>>>>>>> be an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Hi Arun,
Actually , we missed the quotes as shell will consider everything after ";"
as second command.
bin/sqlline.py "localhost;currentSCN=9"
So, can you fix the timestamp for these columns?
Steps you may follow(but request you to try first on non production cluster
as these are not tested by me).
- Take snapshot of your system.catalog( for backup)
- delete such cells from system.catalog at latest timestamp
- set keep_deleted_cells to false in SYSTEM.CATALOG(remember you should not
have any query running and using schema in point in time snapshots).
- do flush and major_compaction on SYSTEM.CATALOG
- when you don't see those columns and open connection at currentSCN=9 and
alter table to add both the columns.
- you may set keep_deleted_cells back to true in SYSTEM.CATALOG
Regards,
Ankit Singhal
Regards,
Ankit Singhal
On Tue, Apr 26, 2016 at 11:26 PM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> James,
>
> Thanks for the answer. But, why was the currentSCN=9 on connection is not
> working in the first place? Do you have any pointers for us to look into?
>
> Thanks,
> Arun
>
> On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
>> Hi Arun,
>> Yes, this could cause problems, as we base the need to upgrade on the
>> time stamp of the system catalog table.
>> Thanks,
>> James
>>
>>
>> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
>> wrote:
>>
>>> Hi Ankit,
>>>
>>> Just following with the question that when the alter statement was
>>> issued with CurrentSCN=9, the current timestamp was not set to 9.
>>> Will this cause an issue in the future if it has to compare the
>>> timestamps?
>>>
>>> Thanks,
>>> Arun
>>>
>>> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> Hi Ankit,
>>>>
>>>> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT
>>>> INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it
>>>> actually added the columns to the SYSTEM.CATALOG.
>>>> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
>>>> timestamp was not set to 9. Will this cause an issue in the future if it
>>>> has to compare the timestamps?
>>>>
>>>> After the alter statement was executed and when queried from HBase, the
>>>> following is the output.
>>>>
>>>> hbase(main):004:0> get
>>>> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
>>>> COLUMN CELL
>>>> 0 ATA_TYPE timestamp=1461362240655,
>>>> value=\x80\x00\x00\x10
>>>> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
>>>> 0:NULLABLE timestamp=1461362240655,
>>>> value=\x80\x00\x00\x01
>>>> 0 RDINAL_POSITION timestamp=1461362240655,
>>>> value=\x80\x00\x004
>>>> 0 ORT_ORDER timestamp=1461362240655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>>
>>>> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <
>>>> ankitsinghal59@gmail.com> wrote:
>>>>
>>>>> Yes Arun, it seems fine to me.
>>>>> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
>>>>> work correctly.(Remember use correct INTEGER byte representation for
>>>>> DATA_TYPE column).
>>>>>
>>>>> And, can you also please share output of
>>>>> > scan 'SYSTEM.SEQUENCE'
>>>>>
>>>>> Regards,
>>>>> Ankit
>>>>>
>>>>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> Ankit,
>>>>>>
>>>>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did the
>>>>>> job.
>>>>>> Is it okay to do the following? These puts in HBase intends to do
>>>>>> what the ALTER command in phoenix supposed to do.
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>>>>
>>>>>> put 'SYSTEM.CATALOG',
>>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Arun
>>>>>>
>>>>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> Ankit,
>>>>>>>
>>>>>>> When tried executing the query, the following exception is thrown.
>>>>>>>
>>>>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR
>>>>>>> 301 (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>>>>
>>>>>>> at
>>>>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>>>>> at
>>>>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>>>>> at
>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>>>>> at
>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>>>>> at
>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>>>>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>>>>> at
>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>>>>> at
>>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>>>>> at sqlline.Commands.execute(Commands.java:822)
>>>>>>> at sqlline.Commands.sql(Commands.java:732)
>>>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>>>>
>>>>>>>
>>>>>>> Thank,
>>>>>>> Arun
>>>>>>>
>>>>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>
>>>>>>>> Arun,
>>>>>>>> As per the system.catalog ,there seems no way that upgrade
>>>>>>>> code(adding new columns and view upgrade for version < v4.5) doesn't
>>>>>>>> execute.
>>>>>>>>
>>>>>>>> But anyways, let's add the required columns manually by following
>>>>>>>> below step;
>>>>>>>>
>>>>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>>>>> >!quit
>>>>>>>>
>>>>>>>> Quit the shell and start new session without CurrentSCN.
>>>>>>>> > ./sqlline.py localhost
>>>>>>>> > !describe system.catalog
>>>>>>>>
>>>>>>>> this should resolve the issue of missing column.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ankit Singhal
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Ankit,James,
>>>>>>>>>
>>>>>>>>> Please find the results you have asked for in the attachment text
>>>>>>>>> document.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Arun,
>>>>>>>>>> the observations you had is understandable.
>>>>>>>>>> So, that's why we again requested the output of
>>>>>>>>>>
>>>>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>>>>> CATALOG*
>>>>>>>>>>
>>>>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>>>>> also see it in !describe command.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>>>>
>>>>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix
>>>>>>>>>>> 4.6.1 client,
>>>>>>>>>>>
>>>>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>>>>
>>>>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP
>>>>>>>>>>> column.
>>>>>>>>>>>
>>>>>>>>>>> Is this an expected behavior?
>>>>>>>>>>>
>>>>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what
>>>>>>>>>>> should I do? This issue is blocking us in the production environment. Any
>>>>>>>>>>> help to resolve or workaround is highly appreciated.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> It's ok if you can just post after grep for CATALOG in a
>>>>>>>>>>>> command output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> One more question to add,
>>>>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have limited the scan in HBase and here is the following
>>>>>>>>>>>>> data.
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>>
>>>>>>>>>>>>> x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>>
>>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>>
>>>>>>>>>>>>> lt
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>>>>> '|org.apache
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' =>
>>>>>>>>>>>>>> 'true'}}
>>>>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER
>>>>>>>>>>>>>> => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS =>
>>>>>>>>>>>>>> '1000',
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Arun,
>>>>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and
>>>>>>>>>>>>>>> post the output back here.
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> James
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>>>>>> column.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> select * from SYSTEM.CATALOG where
>>>>>>>>>>>>>>>> TABLE_NAME=’TEST_TABLE_1’ AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG,
>>>>>>>>>>>>>>>>> by doing !describe on system.catalog.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi
>>>>>>>>>>>>>>>>> <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and
>>>>>>>>>>>>>>>>>> 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran
>>>>>>>>>>>>>>>>>> Sabtharishi <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To add more information on this issue, this happens in
>>>>>>>>>>>>>>>>>>> new phoenix views associated with brand new tables as well. So, this cannot
>>>>>>>>>>>>>>>>>>> be an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
James,
Thanks for the answer. But, why was the currentSCN=9 on connection is not
working in the first place? Do you have any pointers for us to look into?
Thanks,
Arun
On Tue, Apr 26, 2016 at 11:12 AM, James Taylor <ja...@apache.org>
wrote:
> Hi Arun,
> Yes, this could cause problems, as we base the need to upgrade on the time
> stamp of the system catalog table.
> Thanks,
> James
>
>
> On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
>> Hi Ankit,
>>
>> Just following with the question that when the alter statement was issued
>> with CurrentSCN=9, the current timestamp was not set to 9.
>> Will this cause an issue in the future if it has to compare the
>> timestamps?
>>
>> Thanks,
>> Arun
>>
>> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Hi Ankit,
>>>
>>> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT
>>> INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it
>>> actually added the columns to the SYSTEM.CATALOG.
>>> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
>>> timestamp was not set to 9. Will this cause an issue in the future if it
>>> has to compare the timestamps?
>>>
>>> After the alter statement was executed and when queried from HBase, the
>>> following is the output.
>>>
>>> hbase(main):004:0> get
>>> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
>>> COLUMN CELL
>>> 0 ATA_TYPE timestamp=1461362240655,
>>> value=\x80\x00\x00\x10
>>> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
>>> 0:NULLABLE timestamp=1461362240655,
>>> value=\x80\x00\x00\x01
>>> 0 RDINAL_POSITION timestamp=1461362240655,
>>> value=\x80\x00\x004
>>> 0 ORT_ORDER timestamp=1461362240655,
>>> value=\x80\x00\x00\x02
>>>
>>>
>>> Thanks,
>>> Arun
>>>
>>>
>>> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <ankitsinghal59@gmail.com
>>> > wrote:
>>>
>>>> Yes Arun, it seems fine to me.
>>>> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
>>>> work correctly.(Remember use correct INTEGER byte representation for
>>>> DATA_TYPE column).
>>>>
>>>> And, can you also please share output of
>>>> > scan 'SYSTEM.SEQUENCE'
>>>>
>>>> Regards,
>>>> Ankit
>>>>
>>>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Ankit,
>>>>>
>>>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did the
>>>>> job.
>>>>> Is it okay to do the following? These puts in HBase intends to do what
>>>>> the ALTER command in phoenix supposed to do.
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>>>
>>>>> put 'SYSTEM.CATALOG',
>>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Arun
>>>>>
>>>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> Ankit,
>>>>>>
>>>>>> When tried executing the query, the following exception is thrown.
>>>>>>
>>>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
>>>>>> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>>>
>>>>>> at
>>>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>>>> at
>>>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>>>> at
>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>>>> at
>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>>>> at
>>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>>>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>>>> at
>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>>>> at
>>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>>>> at sqlline.Commands.execute(Commands.java:822)
>>>>>> at sqlline.Commands.sql(Commands.java:732)
>>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>>>
>>>>>>
>>>>>> Thank,
>>>>>> Arun
>>>>>>
>>>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>
>>>>>>> Arun,
>>>>>>> As per the system.catalog ,there seems no way that upgrade
>>>>>>> code(adding new columns and view upgrade for version < v4.5) doesn't
>>>>>>> execute.
>>>>>>>
>>>>>>> But anyways, let's add the required columns manually by following
>>>>>>> below step;
>>>>>>>
>>>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>>>> >!quit
>>>>>>>
>>>>>>> Quit the shell and start new session without CurrentSCN.
>>>>>>> > ./sqlline.py localhost
>>>>>>> > !describe system.catalog
>>>>>>>
>>>>>>> this should resolve the issue of missing column.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ankit Singhal
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> Ankit,James,
>>>>>>>>
>>>>>>>> Please find the results you have asked for in the attachment text
>>>>>>>> document.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Arun,
>>>>>>>>> the observations you had is understandable.
>>>>>>>>> So, that's why we again requested the output of
>>>>>>>>>
>>>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>>>> CATALOG*
>>>>>>>>>
>>>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>>>> also see it in !describe command.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>>>
>>>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>>>>>>> client,
>>>>>>>>>>
>>>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>>>
>>>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP
>>>>>>>>>> column.
>>>>>>>>>>
>>>>>>>>>> Is this an expected behavior?
>>>>>>>>>>
>>>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what
>>>>>>>>>> should I do? This issue is blocking us in the production environment. Any
>>>>>>>>>> help to resolve or workaround is highly appreciated.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> One more question to add,
>>>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>>>
>>>>>>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>>
>>>>>>>>>>>> x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>>> value=\x80\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>>
>>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>>
>>>>>>>>>>>> lt
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> James,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>>>> '|org.apache
>>>>>>>>>>>>>
>>>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>>>
>>>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>>>
>>>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>>>
>>>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER
>>>>>>>>>>>>> => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS =>
>>>>>>>>>>>>> '1000',
>>>>>>>>>>>>>
>>>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>>>
>>>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Arun,
>>>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and
>>>>>>>>>>>>>> post the output back here.
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> James
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>>>>> column.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’
>>>>>>>>>>>>>>> AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and
>>>>>>>>>>>>>>>>> 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi
>>>>>>>>>>>>>>>>> <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To add more information on this issue, this happens in
>>>>>>>>>>>>>>>>>> new phoenix views associated with brand new tables as well. So, this cannot
>>>>>>>>>>>>>>>>>> be an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by James Taylor <ja...@apache.org>.
Hi Arun,
Yes, this could cause problems, as we base the need to upgrade on the time
stamp of the system catalog table.
Thanks,
James
On Tuesday, April 26, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
wrote:
> Hi Ankit,
>
> Just following with the question that when the alter statement was issued
> with CurrentSCN=9, the current timestamp was not set to 9.
> Will this cause an issue in the future if it has to compare the timestamps?
>
> Thanks,
> Arun
>
> On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>>
> wrote:
>
>> Hi Ankit,
>>
>> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT
>> INTEGER, IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it
>> actually added the columns to the SYSTEM.CATALOG.
>> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
>> timestamp was not set to 9. Will this cause an issue in the future if it
>> has to compare the timestamps?
>>
>> After the alter statement was executed and when queried from HBase, the
>> following is the output.
>>
>> hbase(main):004:0> get
>> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
>> COLUMN CELL
>> 0 ATA_TYPE timestamp=1461362240655,
>> value=\x80\x00\x00\x10
>> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
>> 0:NULLABLE timestamp=1461362240655,
>> value=\x80\x00\x00\x01
>> 0 RDINAL_POSITION timestamp=1461362240655,
>> value=\x80\x00\x004
>> 0 ORT_ORDER timestamp=1461362240655,
>> value=\x80\x00\x00\x02
>>
>>
>> Thanks,
>> Arun
>>
>>
>> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <ankitsinghal59@gmail.com
>> <javascript:_e(%7B%7D,'cvml','ankitsinghal59@gmail.com');>> wrote:
>>
>>> Yes Arun, it seems fine to me.
>>> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
>>> work correctly.(Remember use correct INTEGER byte representation for
>>> DATA_TYPE column).
>>>
>>> And, can you also please share output of
>>> > scan 'SYSTEM.SEQUENCE'
>>>
>>> Regards,
>>> Ankit
>>>
>>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>>
>>> wrote:
>>>
>>>> Ankit,
>>>>
>>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did the
>>>> job.
>>>> Is it okay to do the following? These puts in HBase intends to do what
>>>> the ALTER command in phoenix supposed to do.
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>>
>>>> put 'SYSTEM.CATALOG',
>>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Arun
>>>>
>>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>
>>>> > wrote:
>>>>
>>>>> Ankit,
>>>>>
>>>>> When tried executing the query, the following exception is thrown.
>>>>>
>>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
>>>>> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>>
>>>>> at
>>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>>> at
>>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>>> at sqlline.Commands.execute(Commands.java:822)
>>>>> at sqlline.Commands.sql(Commands.java:732)
>>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>>
>>>>>
>>>>> Thank,
>>>>> Arun
>>>>>
>>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com
>>>>> <javascript:_e(%7B%7D,'cvml','ankitsinghal59@gmail.com');>> wrote:
>>>>>
>>>>>> Arun,
>>>>>> As per the system.catalog ,there seems no way that upgrade
>>>>>> code(adding new columns and view upgrade for version < v4.5) doesn't
>>>>>> execute.
>>>>>>
>>>>>> But anyways, let's add the required columns manually by following
>>>>>> below step;
>>>>>>
>>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>>> >!quit
>>>>>>
>>>>>> Quit the shell and start new session without CurrentSCN.
>>>>>> > ./sqlline.py localhost
>>>>>> > !describe system.catalog
>>>>>>
>>>>>> this should resolve the issue of missing column.
>>>>>>
>>>>>> Regards,
>>>>>> Ankit Singhal
>>>>>>
>>>>>>
>>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com
>>>>>> <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>> wrote:
>>>>>>
>>>>>>> Ankit,James,
>>>>>>>
>>>>>>> Please find the results you have asked for in the attachment text
>>>>>>> document.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>>> ankitsinghal59@gmail.com
>>>>>>> <javascript:_e(%7B%7D,'cvml','ankitsinghal59@gmail.com');>> wrote:
>>>>>>>
>>>>>>>> Arun,
>>>>>>>> the observations you had is understandable.
>>>>>>>> So, that's why we again requested the output of
>>>>>>>>
>>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>>> CATALOG*
>>>>>>>>
>>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>>> also see it in !describe command.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com
>>>>>>>> <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>> wrote:
>>>>>>>>
>>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>>
>>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>>>>>> client,
>>>>>>>>>
>>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>>
>>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP
>>>>>>>>> column.
>>>>>>>>>
>>>>>>>>> Is this an expected behavior?
>>>>>>>>>
>>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should
>>>>>>>>> I do? This issue is blocking us in the production environment. Any help to
>>>>>>>>> resolve or workaround is highly appreciated.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>>> ankitsinghal59@gmail.com
>>>>>>>>> <javascript:_e(%7B%7D,'cvml','ankitsinghal59@gmail.com');>> wrote:
>>>>>>>>>
>>>>>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com
>>>>>>>>>> <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>> wrote:
>>>>>>>>>>
>>>>>>>>>>> One more question to add,
>>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>>
>>>>>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>>
>>>>>>>>>>> x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x00\x00\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x0C
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>>
>>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>>
>>>>>>>>>>> lt
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com
>>>>>>>>>>> <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> James,
>>>>>>>>>>>>
>>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>>> '|org.apache
>>>>>>>>>>>>
>>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>>
>>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>>
>>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>>
>>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER
>>>>>>>>>>>> => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS =>
>>>>>>>>>>>> '1000',
>>>>>>>>>>>>
>>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>>
>>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>>
>>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>>> jamestaylor@apache.org
>>>>>>>>>>>> <javascript:_e(%7B%7D,'cvml','jamestaylor@apache.org');>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Arun,
>>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and
>>>>>>>>>>>>> post the output back here.
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> James
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>> arun1087@gmail.com
>>>>>>>>>>>>> <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>>>> column.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’
>>>>>>>>>>>>>> AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and
>>>>>>>>>>>>>>>> 4.7 upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi
>>>>>>>>>>>>>>>> <ar...@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Hi Ankit,
Just following with the question that when the alter statement was issued
with CurrentSCN=9, the current timestamp was not set to 9.
Will this cause an issue in the future if it has to compare the timestamps?
Thanks,
Arun
On Mon, Apr 25, 2016 at 10:32 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Hi Ankit,
>
> Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
> the columns to the SYSTEM.CATALOG.
> But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
> timestamp was not set to 9. Will this cause an issue in the future if it
> has to compare the timestamps?
>
> After the alter statement was executed and when queried from HBase, the
> following is the output.
>
> hbase(main):004:0> get
> 'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
> COLUMN CELL
> 0 ATA_TYPE timestamp=1461362240655,
> value=\x80\x00\x00\x10
> 0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
> 0:NULLABLE timestamp=1461362240655,
> value=\x80\x00\x00\x01
> 0 RDINAL_POSITION timestamp=1461362240655,
> value=\x80\x00\x004
> 0 ORT_ORDER timestamp=1461362240655,
> value=\x80\x00\x00\x02
>
>
> Thanks,
> Arun
>
>
> On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
>> Yes Arun, it seems fine to me.
>> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
>> work correctly.(Remember use correct INTEGER byte representation for
>> DATA_TYPE column).
>>
>> And, can you also please share output of
>> > scan 'SYSTEM.SEQUENCE'
>>
>> Regards,
>> Ankit
>>
>> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Ankit,
>>>
>>> Though the ALTER in pheonix failed, a bunch of puts in HBase did the
>>> job.
>>> Is it okay to do the following? These puts in HBase intends to do what
>>> the ALTER command in phoenix supposed to do.
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>>
>>> put 'SYSTEM.CATALOG',
>>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>>
>>>
>>> Thanks,
>>>
>>> Arun
>>>
>>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> Ankit,
>>>>
>>>> When tried executing the query, the following exception is thrown.
>>>>
>>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
>>>> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>>
>>>> at
>>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>>> at
>>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>>> at sqlline.Commands.execute(Commands.java:822)
>>>> at sqlline.Commands.sql(Commands.java:732)
>>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>>
>>>>
>>>> Thank,
>>>> Arun
>>>>
>>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <
>>>> ankitsinghal59@gmail.com> wrote:
>>>>
>>>>> Arun,
>>>>> As per the system.catalog ,there seems no way that upgrade code(adding
>>>>> new columns and view upgrade for version < v4.5) doesn't execute.
>>>>>
>>>>> But anyways, let's add the required columns manually by following
>>>>> below step;
>>>>>
>>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>>> >!quit
>>>>>
>>>>> Quit the shell and start new session without CurrentSCN.
>>>>> > ./sqlline.py localhost
>>>>> > !describe system.catalog
>>>>>
>>>>> this should resolve the issue of missing column.
>>>>>
>>>>> Regards,
>>>>> Ankit Singhal
>>>>>
>>>>>
>>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> Ankit,James,
>>>>>>
>>>>>> Please find the results you have asked for in the attachment text
>>>>>> document.
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>
>>>>>>> Arun,
>>>>>>> the observations you had is understandable.
>>>>>>> So, that's why we again requested the output of
>>>>>>>
>>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>>> CATALOG*
>>>>>>>
>>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>>> also see it in !describe command.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>>
>>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>>>>> client,
>>>>>>>>
>>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>>
>>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>>>>>
>>>>>>>> Is this an expected behavior?
>>>>>>>>
>>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should
>>>>>>>> I do? This issue is blocking us in the production environment. Any help to
>>>>>>>> resolve or workaround is highly appreciated.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> One more question to add,
>>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>>
>>>>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE,
>>>>>>>>>> timestamp=1460354090655, value=u
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>>
>>>>>>>>>> x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>>> type=DeleteFamily
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER,
>>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>>> type=DeleteColumn
>>>>>>>>>>
>>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>>
>>>>>>>>>> lt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> James,
>>>>>>>>>>>
>>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>>> '|org.apache
>>>>>>>>>>>
>>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>>
>>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>>
>>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>>
>>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>>>>>>
>>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>>
>>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>>
>>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Arun,
>>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and
>>>>>>>>>>>> post the output back here.
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> James
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>>> column.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>>
>>>>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’
>>>>>>>>>>>>> AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if not,
>>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Hi Ankit,
Even though the ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
IS_ROW_TIMESTAMP BOOLEAN; statement threw an exception, it actually added
the columns to the SYSTEM.CATALOG.
But, the ./sqlline.py localhost;CurrentSCN=9 did not work. The current
timestamp was not set to 9. Will this cause an issue in the future if it
has to compare the timestamps?
After the alter statement was executed and when queried from HBase, the
following is the output.
hbase(main):004:0> get
'SYSTEM.CATALOG',"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000"
COLUMN CELL
0 ATA_TYPE timestamp=1461362240655,
value=\x80\x00\x00\x10
0:IS_VIEW_REFERENCED timestamp=1461362240655, value=\x00
0:NULLABLE timestamp=1461362240655,
value=\x80\x00\x00\x01
0 RDINAL_POSITION timestamp=1461362240655,
value=\x80\x00\x004
0 ORT_ORDER timestamp=1461362240655,
value=\x80\x00\x00\x02
Thanks,
Arun
On Mon, Apr 25, 2016 at 2:19 AM, Ankit Singhal <an...@gmail.com>
wrote:
> Yes Arun, it seems fine to me.
> Can you do it for BASE_COLUMN_COUNT too so that the dependent features
> work correctly.(Remember use correct INTEGER byte representation for
> DATA_TYPE column).
>
> And, can you also please share output of
> > scan 'SYSTEM.SEQUENCE'
>
> Regards,
> Ankit
>
> On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> Ankit,
>>
>> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
>> Is it okay to do the following? These puts in HBase intends to do what
>> the ALTER command in phoenix supposed to do.
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>>
>> put 'SYSTEM.CATALOG',
>> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>>
>>
>> Thanks,
>>
>> Arun
>>
>> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Ankit,
>>>
>>> When tried executing the query, the following exception is thrown.
>>>
>>> Error: ERROR 301 (23000): Concurrent modification to table.
>>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
>>> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>>
>>> at
>>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>>> at
>>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>>> at sqlline.Commands.execute(Commands.java:822)
>>> at sqlline.Commands.sql(Commands.java:732)
>>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>>> at sqlline.SqlLine.begin(SqlLine.java:681)
>>> at sqlline.SqlLine.start(SqlLine.java:398)
>>>
>>>
>>> Thank,
>>> Arun
>>>
>>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <ankitsinghal59@gmail.com
>>> > wrote:
>>>
>>>> Arun,
>>>> As per the system.catalog ,there seems no way that upgrade code(adding
>>>> new columns and view upgrade for version < v4.5) doesn't execute.
>>>>
>>>> But anyways, let's add the required columns manually by following below
>>>> step;
>>>>
>>>> > ./sqlline.py localhost;CurrentSCN=9
>>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>>> IS_ROW_TIMESTAMP BOOLEAN;
>>>> >!quit
>>>>
>>>> Quit the shell and start new session without CurrentSCN.
>>>> > ./sqlline.py localhost
>>>> > !describe system.catalog
>>>>
>>>> this should resolve the issue of missing column.
>>>>
>>>> Regards,
>>>> Ankit Singhal
>>>>
>>>>
>>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Ankit,James,
>>>>>
>>>>> Please find the results you have asked for in the attachment text
>>>>> document.
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>
>>>>>> Arun,
>>>>>> the observations you had is understandable.
>>>>>> So, that's why we again requested the output of
>>>>>>
>>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep
>>>>>> CATALOG*
>>>>>>
>>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>>> also see it in !describe command.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>>
>>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>>>> client,
>>>>>>>
>>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>>
>>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>>>>
>>>>>>> Is this an expected behavior?
>>>>>>>
>>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>>>>> do? This issue is blocking us in the production environment. Any help to
>>>>>>> resolve or workaround is highly appreciated.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>
>>>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> One more question to add,
>>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>>
>>>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>>>>> value=u
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>>
>>>>>>>>> x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>> value=\x80\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>>> type=DeleteFamily
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>>> type=DeleteColumn
>>>>>>>>>
>>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>>
>>>>>>>>> lt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> James,
>>>>>>>>>>
>>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>>> '|org.apache
>>>>>>>>>>
>>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>>> coprocessor$3 =>
>>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>>
>>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>>
>>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>>
>>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>>>>>
>>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>>
>>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>>
>>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Arun,
>>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and
>>>>>>>>>>> post the output back here.
>>>>>>>>>>> Thanks,
>>>>>>>>>>> James
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>>> column.
>>>>>>>>>>>>
>>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>>
>>>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’
>>>>>>>>>>>> AND TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> if not,
>>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>>
>>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Yes Arun, it seems fine to me.
Can you do it for BASE_COLUMN_COUNT too so that the dependent features work
correctly.(Remember use correct INTEGER byte representation for DATA_TYPE
column).
And, can you also please share output of
> scan 'SYSTEM.SEQUENCE'
Regards,
Ankit
On Fri, Apr 22, 2016 at 9:14 PM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Ankit,
>
> Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
> Is it okay to do the following? These puts in HBase intends to do what the
> ALTER command in phoenix supposed to do.
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
>
> put 'SYSTEM.CATALOG',
> "\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
>
>
> Thanks,
>
> Arun
>
> On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> Ankit,
>>
>> When tried executing the query, the following exception is thrown.
>>
>> Error: ERROR 301 (23000): Concurrent modification to table.
>> tableName=SYSTEM.CATALOG (state=23000,code=301)
>> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
>> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>>
>> at
>> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
>> at
>> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
>> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
>> at sqlline.Commands.execute(Commands.java:822)
>> at sqlline.Commands.sql(Commands.java:732)
>> at sqlline.SqlLine.dispatch(SqlLine.java:808)
>> at sqlline.SqlLine.begin(SqlLine.java:681)
>> at sqlline.SqlLine.start(SqlLine.java:398)
>>
>>
>> Thank,
>> Arun
>>
>> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
>> wrote:
>>
>>> Arun,
>>> As per the system.catalog ,there seems no way that upgrade code(adding
>>> new columns and view upgrade for version < v4.5) doesn't execute.
>>>
>>> But anyways, let's add the required columns manually by following below
>>> step;
>>>
>>> > ./sqlline.py localhost;CurrentSCN=9
>>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>>> IS_ROW_TIMESTAMP BOOLEAN;
>>> >!quit
>>>
>>> Quit the shell and start new session without CurrentSCN.
>>> > ./sqlline.py localhost
>>> > !describe system.catalog
>>>
>>> this should resolve the issue of missing column.
>>>
>>> Regards,
>>> Ankit Singhal
>>>
>>>
>>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> Ankit,James,
>>>>
>>>> Please find the results you have asked for in the attachment text
>>>> document.
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <
>>>> ankitsinghal59@gmail.com> wrote:
>>>>
>>>>> Arun,
>>>>> the observations you had is understandable.
>>>>> So, that's why we again requested the output of
>>>>>
>>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>>>>
>>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>>> system.catalog table so that your phoenix setup works properly and you can
>>>>> also see it in !describe command.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>>
>>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>>> client,
>>>>>>
>>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>>
>>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>>>
>>>>>> Is this an expected behavior?
>>>>>>
>>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>>>> do? This issue is blocking us in the production environment. Any help to
>>>>>> resolve or workaround is highly appreciated.
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>
>>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> One more question to add,
>>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>>
>>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>>> timestamp=1460354090655, value=default
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>>>>> value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>>>>> value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>>>> value=u
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>>
>>>>>>>> x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>> value=\x80\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>>> type=DeleteFamily
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x01
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>>> value=\x80\x00\x00\x02
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>>> type=DeleteColumn
>>>>>>>>
>>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>>
>>>>>>>> lt
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Arun
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> James,
>>>>>>>>>
>>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>>> '|org.apache
>>>>>>>>>
>>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>>> coprocessor$3 =>
>>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>>
>>>>>>>>> |', coprocessor$4 =>
>>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>>
>>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>>
>>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>>>>
>>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>>
>>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>>
>>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Arun,
>>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and post
>>>>>>>>>> the output back here.
>>>>>>>>>> Thanks,
>>>>>>>>>> James
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>>> column.
>>>>>>>>>>>
>>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>>
>>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>>
>>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> if not,
>>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>>
>>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert
>>>>>>>>>>>>> query SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but
>>>>>>>>>>>>> PTableImpl.getColumn() is failing with error:"Undefined column.
>>>>>>>>>>>>> columnName=IS_ROW_TIMESTAMP" . Does this mean that PTableImpl is reading
>>>>>>>>>>>>> from cached entity of SYSTEM.CATALOG before 4.6 upgrade?"
>>>>>>>>>>>>>
>>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> James,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Ankit,
Though the ALTER in pheonix failed, a bunch of puts in HBase did the job.
Is it okay to do the following? These puts in HBase intends to do what the
ALTER command in phoenix supposed to do.
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:DATA_TYPE',"\x80\x00\x00\x10",9
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:IS_VIEW_REFERENCED',"\x00",9
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:NULLABLE',"\x80\x00\x00\x01",9
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:ORDINAL_POSITION',"\x80\x00\x001",9
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:SORT_ORDER',"\x80\x00\x00\x02",9
put 'SYSTEM.CATALOG',
"\x00SYSTEM\x00CATALOG\x00IS_ROW_TIMESTAMP\x000",'0:_0','',9
Thanks,
Arun
On Fri, Apr 22, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Ankit,
>
> When tried executing the query, the following exception is thrown.
>
> Error: ERROR 301 (23000): Concurrent modification to table.
> tableName=SYSTEM.CATALOG (state=23000,code=301)
> org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
> (23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
>
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
>
>
> Thank,
> Arun
>
> On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
>> Arun,
>> As per the system.catalog ,there seems no way that upgrade code(adding
>> new columns and view upgrade for version < v4.5) doesn't execute.
>>
>> But anyways, let's add the required columns manually by following below
>> step;
>>
>> > ./sqlline.py localhost;CurrentSCN=9
>> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
>> IS_ROW_TIMESTAMP BOOLEAN;
>> >!quit
>>
>> Quit the shell and start new session without CurrentSCN.
>> > ./sqlline.py localhost
>> > !describe system.catalog
>>
>> this should resolve the issue of missing column.
>>
>> Regards,
>> Ankit Singhal
>>
>>
>> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Ankit,James,
>>>
>>> Please find the results you have asked for in the attachment text
>>> document.
>>>
>>> Thanks,
>>> Arun
>>>
>>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <ankitsinghal59@gmail.com
>>> > wrote:
>>>
>>>> Arun,
>>>> the observations you had is understandable.
>>>> So, that's why we again requested the output of
>>>>
>>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>>>
>>>> So that we can help you in adding ROW_TIMESTAMP column in
>>>> system.catalog table so that your phoenix setup works properly and you can
>>>> also see it in !describe command.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>>
>>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>>> client,
>>>>>
>>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>>
>>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>>
>>>>> Is this an expected behavior?
>>>>>
>>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>>> do? This issue is blocking us in the production environment. Any help to
>>>>> resolve or workaround is highly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>>
>>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>
>>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> One more question to add,
>>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>>
>>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x05
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>>> timestamp=1460354090655, value=default
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>>> timestamp=1460354090655, value=\x01
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>>>> value=\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>>> value=u
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>>
>>>>>>> x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> value=\x80\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>>> type=DeleteFamily
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x0C
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>>> timestamp=1460354090655, value=\x00
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x01
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>>> value=\x80\x00\x00\x02
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT,
>>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>>> type=DeleteColumn
>>>>>>>
>>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>>
>>>>>>> lt
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> James,
>>>>>>>>
>>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>>> '|org.apache
>>>>>>>>
>>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>>> coprocessor$3 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>>
>>>>>>>> |', coprocessor$4 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>>
>>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>>
>>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>>>
>>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS =>
>>>>>>>> 'true', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>>
>>>>>>>> 1 row in 0.6060 seconds
>>>>>>>>
>>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Arun,
>>>>>>>>> Please run the command Ankit mentioned in an HBase shell and post
>>>>>>>>> the output back here.
>>>>>>>>> Thanks,
>>>>>>>>> James
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP
>>>>>>>>>> column.
>>>>>>>>>>
>>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>>
>>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Arun,
>>>>>>>>>>>
>>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by
>>>>>>>>>>> doing !describe on system.catalog.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> if not,
>>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>>
>>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ankit Singhal
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>>>>>>> before 4.6 upgrade?"
>>>>>>>>>>>>
>>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> James,
>>>>>>>>>>>>>
>>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Arun
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Ankit,
When tried executing the query, the following exception is thrown.
Error: ERROR 301 (23000): Concurrent modification to table.
tableName=SYSTEM.CATALOG (state=23000,code=301)
org.apache.phoenix.schema.ConcurrentTableMutationException: ERROR 301
(23000): Concurrent modification to table. tableName=SYSTEM.CATALOG
at
org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:2339)
at
org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:2641)
at
org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1106)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:322)
at
org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:314)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:312)
at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1435)
at sqlline.Commands.execute(Commands.java:822)
at sqlline.Commands.sql(Commands.java:732)
at sqlline.SqlLine.dispatch(SqlLine.java:808)
at sqlline.SqlLine.begin(SqlLine.java:681)
at sqlline.SqlLine.start(SqlLine.java:398)
Thank,
Arun
On Fri, Apr 22, 2016 at 8:59 AM, Ankit Singhal <an...@gmail.com>
wrote:
> Arun,
> As per the system.catalog ,there seems no way that upgrade code(adding new
> columns and view upgrade for version < v4.5) doesn't execute.
>
> But anyways, let's add the required columns manually by following below
> step;
>
> > ./sqlline.py localhost;CurrentSCN=9
> > ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
> IS_ROW_TIMESTAMP BOOLEAN;
> >!quit
>
> Quit the shell and start new session without CurrentSCN.
> > ./sqlline.py localhost
> > !describe system.catalog
>
> this should resolve the issue of missing column.
>
> Regards,
> Ankit Singhal
>
>
> On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> Ankit,James,
>>
>> Please find the results you have asked for in the attachment text
>> document.
>>
>> Thanks,
>> Arun
>>
>> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
>> wrote:
>>
>>> Arun,
>>> the observations you had is understandable.
>>> So, that's why we again requested the output of
>>>
>>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>>
>>> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
>>> table so that your phoenix setup works properly and you can also see it in
>>> !describe command.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>>
>>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1
>>>> client,
>>>>
>>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>>
>>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>>
>>>> Is this an expected behavior?
>>>>
>>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I
>>>> do? This issue is blocking us in the production environment. Any help to
>>>> resolve or workaround is highly appreciated.
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>>
>>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>>> ankitsinghal59@gmail.com> wrote:
>>>>
>>>>> It's ok if you can just post after grep for CATALOG in a command
>>>>> output (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>>
>>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> One more question to add,
>>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>>
>>>>>> I have limited the scan in HBase and here is the following data.
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x05
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>>> timestamp=1460354090655, value=default
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>>> timestamp=1460354090655, value=\x01
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>>> value=\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>>> value=u
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>>
>>>>>> x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> value=\x80\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_PK
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>>> type=DeleteFamily
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x0C
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS,
>>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>>> timestamp=1460354090655, value=\x00
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x01
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>>> value=\x80\x00\x00\x02
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>>> type=DeleteColumn
>>>>>>
>>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>>
>>>>>> lt
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> James,
>>>>>>>
>>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>>> '|org.apache
>>>>>>>
>>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>>> coprocessor$3 =>
>>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>>
>>>>>>> |', coprocessor$4 =>
>>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>>
>>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>>
>>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>>
>>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>>>>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>>
>>>>>>> 1 row in 0.6060 seconds
>>>>>>>
>>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>>> jamestaylor@apache.org> wrote:
>>>>>>>
>>>>>>>> Arun,
>>>>>>>> Please run the command Ankit mentioned in an HBase shell and post
>>>>>>>> the output back here.
>>>>>>>> Thanks,
>>>>>>>> James
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>>>>>
>>>>>>>>> But we do see this column from select statement:
>>>>>>>>>
>>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Arun,
>>>>>>>>>>
>>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>>>>>> !describe on system.catalog.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> if not,
>>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>>
>>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ankit Singhal
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>>>>>> before 4.6 upgrade?"
>>>>>>>>>>>
>>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> James,
>>>>>>>>>>>>
>>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Arun
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Arun,
As per the system.catalog ,there seems no way that upgrade code(adding new
columns and view upgrade for version < v4.5) doesn't execute.
But anyways, let's add the required columns manually by following below
step;
> ./sqlline.py localhost;CurrentSCN=9
> ALTER TABLE SYSTEM.CATALOG ADD BASE_COLUMN_COUNT INTEGER,
IS_ROW_TIMESTAMP BOOLEAN;
>!quit
Quit the shell and start new session without CurrentSCN.
> ./sqlline.py localhost
> !describe system.catalog
this should resolve the issue of missing column.
Regards,
Ankit Singhal
On Fri, Apr 22, 2016 at 3:02 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Ankit,James,
>
> Please find the results you have asked for in the attachment text document.
>
> Thanks,
> Arun
>
> On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
> wrote:
>
>> Arun,
>> the observations you had is understandable.
>> So, that's why we again requested the output of
>>
>> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>>
>> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
>> table so that your phoenix setup works properly and you can also see it in
>> !describe command.
>>
>>
>>
>>
>>
>>
>> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>>
>>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>>>
>>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>>
>>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>>
>>> Is this an expected behavior?
>>>
>>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
>>> This issue is blocking us in the production environment. Any help to
>>> resolve or workaround is highly appreciated.
>>>
>>> Thanks,
>>> Arun
>>>
>>>
>>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <
>>> ankitsinghal59@gmail.com> wrote:
>>>
>>>> It's ok if you can just post after grep for CATALOG in a command output
>>>> (scan 'SYSTEM.CATALOG', {RAW=>true}).
>>>>
>>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> One more question to add,
>>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>>
>>>>> I have limited the scan in HBase and here is the following data.
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x05
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>>> timestamp=1460354090655, value=default
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>>> value=\x00
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
>>>>> value=\x00
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>>> value=\x00
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>>> timestamp=1460354090655, value=\x01
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>>> value=\x00
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>>> value=u
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x0C
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>>
>>>>> x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x0C
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>> value=\x80\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_PK
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x0C
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x0C
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>>> type=DeleteFamily
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME,
>>>>> timestamp=1460354090655, type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x0C
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>>> timestamp=1460354090655, value=\x00
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x01
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>>> value=\x80\x00\x00\x02
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>>> type=DeleteColumn
>>>>>
>>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>>
>>>>> lt
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Arun
>>>>>
>>>>>
>>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> James,
>>>>>>
>>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>>> '|org.apache
>>>>>>
>>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>>> coprocessor$3 =>
>>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>>
>>>>>> |', coprocessor$4 =>
>>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>>
>>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>>
>>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>>> COLUMN FAMILIES DESCRIPTION
>>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>>
>>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>>>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>>
>>>>>> 1 row in 0.6060 seconds
>>>>>>
>>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <
>>>>>> jamestaylor@apache.org> wrote:
>>>>>>
>>>>>>> Arun,
>>>>>>> Please run the command Ankit mentioned in an HBase shell and post
>>>>>>> the output back here.
>>>>>>> Thanks,
>>>>>>> James
>>>>>>>
>>>>>>>
>>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>>>>
>>>>>>>> But we do see this column from select statement:
>>>>>>>>
>>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Arun
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Arun,
>>>>>>>>>
>>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>>>>> !describe on system.catalog.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> if not,
>>>>>>>>> can you share the output of below command. As it seems
>>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>>
>>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ankit Singhal
>>>>>>>>>
>>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>>>>> before 4.6 upgrade?"
>>>>>>>>>>
>>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> James,
>>>>>>>>>>>
>>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>>> could be approached from this point?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Arun
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Ankit,James,
Please find the results you have asked for in the attachment text document.
Thanks,
Arun
On Thu, Apr 21, 2016 at 3:12 AM, Ankit Singhal <an...@gmail.com>
wrote:
> Arun,
> the observations you had is understandable.
> So, that's why we again requested the output of
>
> *echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
>
> So that we can help you in adding ROW_TIMESTAMP column in system.catalog
> table so that your phoenix setup works properly and you can also see it in
> !describe command.
>
>
>
>
>
>
> On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>>
>> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>>
>> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>>
>> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>>
>> Is this an expected behavior?
>>
>> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
>> This issue is blocking us in the production environment. Any help to
>> resolve or workaround is highly appreciated.
>>
>> Thanks,
>> Arun
>>
>>
>> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <ankitsinghal59@gmail.com
>> > wrote:
>>
>>> It's ok if you can just post after grep for CATALOG in a command output (scan
>>> 'SYSTEM.CATALOG', {RAW=>true}).
>>>
>>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> One more question to add,
>>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>>
>>>> I have limited the scan in HBase and here is the following data.
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x05
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>>> timestamp=1460354090655, value=default
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>>> value=\x00
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
>>>> value=\x00
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>>> value=\x00
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>>> timestamp=1460354090655, value=\x01
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>>> value=\x00
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>>> value=u
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x0C
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x01
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>> timestamp=1460354090655, value=\x80\x00\x00\x04
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>>
>>>> x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x0C
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>> value=\x80\x01
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x00
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>> timestamp=1460354090655, value=\x80\x00\x00\x05
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_PK
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x0C
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x01
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>> timestamp=1460354090655, value=\x80\x00\x00\x01
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x0C
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x01
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>> timestamp=1460354090655, value=\x80\x00\x00\x03
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>>> type=DeleteFamily
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x0C
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>>> timestamp=1460354090655, value=\x00
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x01
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION,
>>>> timestamp=1460354090655, value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>>> value=\x80\x00\x00\x02
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>>> type=DeleteColumn
>>>>
>>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>>
>>>> lt
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Arun
>>>>
>>>>
>>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> James,
>>>>>
>>>>> Table SYSTEM.CATALOG is ENABLED
>>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>>> '|org.apache
>>>>>
>>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>>> coprocessor$3 =>
>>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>>
>>>>> |', coprocessor$4 =>
>>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>>
>>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>>
>>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>>> COLUMN FAMILIES DESCRIPTION
>>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER =>
>>>>> 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>>
>>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>>
>>>>> 1 row in 0.6060 seconds
>>>>>
>>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>>
>>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <jamestaylor@apache.org
>>>>> > wrote:
>>>>>
>>>>>> Arun,
>>>>>> Please run the command Ankit mentioned in an HBase shell and post the
>>>>>> output back here.
>>>>>> Thanks,
>>>>>> James
>>>>>>
>>>>>>
>>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>>>
>>>>>>> But we do see this column from select statement:
>>>>>>>
>>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Arun
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Arun,
>>>>>>>>
>>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>>>> !describe on system.catalog.
>>>>>>>>
>>>>>>>>
>>>>>>>> if not,
>>>>>>>> can you share the output of below command. As it seems
>>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>>
>>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ankit Singhal
>>>>>>>>
>>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>>>> before 4.6 upgrade?"
>>>>>>>>>
>>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> James,
>>>>>>>>>>
>>>>>>>>>> To add more information on this issue, this happens in new
>>>>>>>>>> phoenix views associated with brand new tables as well. So, this cannot be
>>>>>>>>>> an upgrade/migration issue. Not figured out a specific way to reproduce
>>>>>>>>>> this issue yet. Could you throw some ideas on what direction this problem
>>>>>>>>>> could be approached from this point?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Arun
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Arun,
the observations you had is understandable.
So, that's why we again requested the output of
*echo "scan 'SYSTEM.CATALOG',{RAW=>true}"|bin/hbase shell|grep CATALOG*
So that we can help you in adding ROW_TIMESTAMP column in system.catalog
table so that your phoenix setup works properly and you can also see it in
!describe command.
On Thu, Apr 21, 2016 at 4:20 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
>
> In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
>
> !describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
>
> But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
>
> Is this an expected behavior?
>
> If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
> This issue is blocking us in the production environment. Any help to
> resolve or workaround is highly appreciated.
>
> Thanks,
> Arun
>
>
> On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
> wrote:
>
>> It's ok if you can just post after grep for CATALOG in a command output (scan
>> 'SYSTEM.CATALOG', {RAW=>true}).
>>
>> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> One more question to add,
>>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>>
>>> I have limited the scan in HBase and here is the following data.
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT,
>>> timestamp=1460354090655, value=\x7F\xFF\xFF\xFF
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>>> value=\x80\x00\x00\x05
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>>> timestamp=1460354090655, value=default
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>>> timestamp=1460354090655, value=\x01
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655,
>>> value=u
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x0C
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>> timestamp=1460354090655, value=\x00
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x01
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>>> value=\x80\x00\x00\x04
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>>
>>> x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x0C
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>> timestamp=1460354090655, value=\x00
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>> value=\x80\x01
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x00
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>>> value=\x80\x00\x00\x05
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_PK
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x0C
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>> timestamp=1460354090655, value=\x00
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x01
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>>> value=\x80\x00\x00\x01
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x0C
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>> timestamp=1460354090655, value=\x00
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x01
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>>> value=\x80\x00\x00\x03
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>>
>>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842,
>>> type=DeleteFamily
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x0C
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>>> value=\x00
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>>> timestamp=1460354090655, value=\x00
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>>> value=\x80\x00\x00\x01
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>>> value=\x80\x00\x00\x02
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>>> type=DeleteColumn
>>>
>>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>>
>>> lt
>>>
>>>
>>> Thanks,
>>>
>>> Arun
>>>
>>>
>>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> James,
>>>>
>>>> Table SYSTEM.CATALOG is ENABLED
>>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>>> '|org.apache
>>>>
>>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|',
>>>> coprocessor$3 =>
>>>> '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>>
>>>> |', coprocessor$4 =>
>>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>>
>>>> taEndpointImpl|1|', coprocessor$6 =>
>>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>>> {'SPLIT_POLICY' => 'org.apache.
>>>>
>>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>>> COLUMN FAMILIES DESCRIPTION
>>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
>>>> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>>
>>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>>
>>>> 1 row in 0.6060 seconds
>>>>
>>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>>
>>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
>>>> wrote:
>>>>
>>>>> Arun,
>>>>> Please run the command Ankit mentioned in an HBase shell and post the
>>>>> output back here.
>>>>> Thanks,
>>>>> James
>>>>>
>>>>>
>>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>>
>>>>>> But we do see this column from select statement:
>>>>>>
>>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Arun
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Arun,
>>>>>>>
>>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>>> !describe on system.catalog.
>>>>>>>
>>>>>>>
>>>>>>> if not,
>>>>>>> can you share the output of below command. As it seems
>>>>>>> SYSTEM.CATALOG was updated with timestamp greater v4.6 timestamp , and
>>>>>>> which stopping upgrade code to add a new column.
>>>>>>>
>>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ankit Singhal
>>>>>>>
>>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>>> before 4.6 upgrade?"
>>>>>>>>
>>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> James,
>>>>>>>>>
>>>>>>>>> To add more information on this issue, this happens in new phoenix
>>>>>>>>> views associated with brand new tables as well. So, this cannot be an
>>>>>>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>>>>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>>>>>>> be approached from this point?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Arun
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
Another observation:(After upgrading from Phoenix 4.4 to 4.6.1)
In a new SYSTEM.CATALOG table , when connected from phoenix 4.6.1 client,
!describe SYSTEM.CATALOG does not show IS_ROW_TIMESTAMP
But, select * from SYSTEM.CATALOG shows the IS_ROW_TIMESTAMP column.
Is this an expected behavior?
If I have to see IS_ROW_TIMESTAMP when doing describe, what should I do?
This issue is blocking us in the production environment. Any help to
resolve or workaround is highly appreciated.
Thanks,
Arun
On Wed, Apr 20, 2016 at 12:01 PM, Ankit Singhal <an...@gmail.com>
wrote:
> It's ok if you can just post after grep for CATALOG in a command output (scan
> 'SYSTEM.CATALOG', {RAW=>true}).
>
> On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> One more question to add,
>> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>>
>> I have limited the scan in HBase and here is the following data.
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
>> value=\x7F\xFF\xFF\xFF
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
>> value=\x80\x00\x00\x05
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
>> timestamp=1460354090655, value=default
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
>> timestamp=1460354090655, value=\x01
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
>> value=\x80\x00\x00\x00\x00\x00\x00\x00
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>> value=\x80\x00\x00\x0C
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>> timestamp=1460354090655, value=\x00
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>> value=\x80\x00\x00\x01
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>> value=\x80\x00\x00\x04
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>>
>> x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>> value=\x80\x00\x00\x0C
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>> timestamp=1460354090655, value=\x00
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>> value=\x80\x01
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>> value=\x80\x00\x00\x00
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>> value=\x80\x00\x00\x05
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_PK
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>> value=\x80\x00\x00\x0C
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>> timestamp=1460354090655, value=\x00
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>> value=\x80\x00\x00\x01
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>> value=\x80\x00\x00\x01
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>> value=\x80\x00\x00\x0C
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>> timestamp=1460354090655, value=\x00
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>> value=\x80\x00\x00\x01
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>> value=\x80\x00\x00\x03
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>>
>> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
>> value=\x80\x00\x00\x0C
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
>> value=\x00
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED,
>> timestamp=1460354090655, value=\x00
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
>> value=\x80\x00\x00\x01
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
>> value=\x80\x00\x00\x02
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
>> type=DeleteColumn
>>
>> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>>
>> lt
>>
>>
>> Thanks,
>>
>> Arun
>>
>>
>> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> James,
>>>
>>> Table SYSTEM.CATALOG is ENABLED
>>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>>> '|org.apache
>>>
>>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
>>> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>>
>>> |', coprocessor$4 =>
>>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>>
>>> taEndpointImpl|1|', coprocessor$6 =>
>>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>>> {'SPLIT_POLICY' => 'org.apache.
>>>
>>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>>> COLUMN FAMILIES DESCRIPTION
>>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
>>> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>>
>>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>>
>>> 1 row in 0.6060 seconds
>>>
>>> The above is for describe SYSTEM.CATALOG. The output for scan
>>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>>
>>> Thanks,
>>> Arun
>>>
>>>
>>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
>>> wrote:
>>>
>>>> Arun,
>>>> Please run the command Ankit mentioned in an HBase shell and post the
>>>> output back here.
>>>> Thanks,
>>>> James
>>>>
>>>>
>>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>>
>>>>> But we do see this column from select statement:
>>>>>
>>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Arun
>>>>>
>>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>>> ankitsinghal59@gmail.com> wrote:
>>>>>
>>>>>> Hi Arun,
>>>>>>
>>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>>> !describe on system.catalog.
>>>>>>
>>>>>>
>>>>>> if not,
>>>>>> can you share the output of below command. As it seems SYSTEM.CATALOG
>>>>>> was updated with timestamp greater v4.6 timestamp , and which stopping
>>>>>> upgrade code to add a new column.
>>>>>>
>>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Ankit Singhal
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>>> before 4.6 upgrade?"
>>>>>>>
>>>>>>> We do see that clearCache() is being called for 4.7, and 4.7
>>>>>>> upgrades from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>>> arun1087@gmail.com> wrote:
>>>>>>>
>>>>>>>> James,
>>>>>>>>
>>>>>>>> To add more information on this issue, this happens in new phoenix
>>>>>>>> views associated with brand new tables as well. So, this cannot be an
>>>>>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>>>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>>>>>> be approached from this point?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Arun
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
It's ok if you can just post after grep for CATALOG in a command output (scan
'SYSTEM.CATALOG', {RAW=>true}).
On Wed, Apr 20, 2016 at 10:07 PM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> One more question to add,
> Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
>
> I have limited the scan in HBase and here is the following data.
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
> value=\x7F\xFF\xFF\xFF
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
> timestamp=1460354090655, value=default
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
> timestamp=1460354090655, value=\x01
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
> value=\x80\x00\x00\x00\x00\x00\x00\x00
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x04
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
>
> x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> value=\x80\x01
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x00
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x05
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_PK
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_SUB_ORG_ID\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x03
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_BY\x00default
>
> \x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
> value=\x80\x00\x00\x0C
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
> value=\x00
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
> value=\x80\x00\x00\x01
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
> value=\x80\x00\x00\x02
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
> \x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655,
> type=DeleteColumn
>
> _0_1460354090089\x00CDS_UPDATED_DATE\x00defau
>
> lt
>
>
> Thanks,
>
> Arun
>
>
> On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> James,
>>
>> Table SYSTEM.CATALOG is ENABLED
>> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
>> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
>> '|org.apache
>>
>> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
>> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>>
>> |', coprocessor$4 =>
>> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
>> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>>
>> taEndpointImpl|1|', coprocessor$6 =>
>> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
>> {'SPLIT_POLICY' => 'org.apache.
>>
>> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
>> COLUMN FAMILIES DESCRIPTION
>> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
>> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>>
>> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
>> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>>
>> 1 row in 0.6060 seconds
>>
>> The above is for describe SYSTEM.CATALOG. The output for scan
>> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>>
>> Thanks,
>> Arun
>>
>>
>> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
>> wrote:
>>
>>> Arun,
>>> Please run the command Ankit mentioned in an HBase shell and post the
>>> output back here.
>>> Thanks,
>>> James
>>>
>>>
>>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>>
>>>> But we do see this column from select statement:
>>>>
>>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Arun
>>>>
>>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <
>>>> ankitsinghal59@gmail.com> wrote:
>>>>
>>>>> Hi Arun,
>>>>>
>>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>>> !describe on system.catalog.
>>>>>
>>>>>
>>>>> if not,
>>>>> can you share the output of below command. As it seems SYSTEM.CATALOG
>>>>> was updated with timestamp greater v4.6 timestamp , and which stopping
>>>>> upgrade code to add a new column.
>>>>>
>>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Ankit Singhal
>>>>>
>>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> After further investigation, we found that Phoenix Upsert query
>>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>>> before 4.6 upgrade?"
>>>>>>
>>>>>> We do see that clearCache() is being called for 4.7, and 4.7 upgrades
>>>>>> from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>>> arun1087@gmail.com> wrote:
>>>>>>
>>>>>>> James,
>>>>>>>
>>>>>>> To add more information on this issue, this happens in new phoenix
>>>>>>> views associated with brand new tables as well. So, this cannot be an
>>>>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>>>>> be approached from this point?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Arun
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
One more question to add,
Do we need to have 1000 versions, and KEEP_DELETED_CELL=true?
I have limited the scan in HBase and here is the following data.
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:BASE_COLUMN_COUNT, timestamp=1460354090655,
value=\x7F\xFF\xFF\xFF
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:COLUMN_COUNT, timestamp=1460354090655,
value=\x80\x00\x00\x05
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DEFAULT_COLUMN_FAMILY,
timestamp=1460354090655, value=default
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:DISABLE_WAL, timestamp=1460354090655,
value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:IMMUTABLE_ROWS, timestamp=1460354090655,
value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_STATE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:INDEX_TYPE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:MULTI_TENANT, timestamp=1460354090655,
value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:ROW_KEY_ORDER_OPTIMIZABLE,
timestamp=1460354090655, value=\x01
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:SALT_BUCKETS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:STORE_NULLS, timestamp=1460354090655,
value=\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_SEQ_NUM, timestamp=1460354090655,
value=\x80\x00\x00\x00\x00\x00\x00\x00
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:TABLE_TYPE, timestamp=1460354090655, value=u
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_INDEX_ID, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_STATEMENT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:VIEW_TYPE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
value=\x80\x00\x00\x04
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_CLIENT_CORELATION_ID\
x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
value=\x80\x01
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
value=\x80\x00\x00\x00
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
value=\x80\x00\x00\x05
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_PK
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_SUB_ORG_ID\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
value=\x80\x00\x00\x03
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_BY\x00default
\x00\x00TEST_TABLE_2 column=0:, timestamp=1460455162842, type=DeleteFamily
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ARRAY_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_DEF, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:COLUMN_SIZE, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TABLE_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DATA_TYPE, timestamp=1460354090655,
value=\x80\x00\x00\x0C
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:DECIMAL_DIGITS, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_ROW_TIMESTAMP, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:IS_VIEW_REFERENCED, timestamp=1460354090655,
value=\x00
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:KEY_SEQ, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:NULLABLE, timestamp=1460354090655,
value=\x80\x00\x00\x01
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:ORDINAL_POSITION, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:PK_NAME, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:SORT_ORDER, timestamp=1460354090655,
value=\x80\x00\x00\x02
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:VIEW_CONSTANT, timestamp=1460354090655,
type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
\x00\x00TEST_TABLE_2 column=0:_0, timestamp=1460354090655, type=DeleteColumn
_0_1460354090089\x00CDS_UPDATED_DATE\x00defau
lt
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:31 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> James,
>
> Table SYSTEM.CATALOG is ENABLED
> SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
> '|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
> '|org.apache
>
> .phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3
> => '|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
>
> |', coprocessor$4 =>
> '|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
> coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
>
> taEndpointImpl|1|', coprocessor$6 =>
> '|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
> {'SPLIT_POLICY' => 'org.apache.
>
> phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
> COLUMN FAMILIES DESCRIPTION
> {NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
>
> MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
> BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
>
> 1 row in 0.6060 seconds
>
> The above is for describe SYSTEM.CATALOG. The output for scan
> 'SYSTEM.CATALOG', {RAW=>true} is too huge.
>
> Thanks,
> Arun
>
>
> On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
> wrote:
>
>> Arun,
>> Please run the command Ankit mentioned in an HBase shell and post the
>> output back here.
>> Thanks,
>> James
>>
>>
>> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>>
>>> But we do see this column from select statement:
>>>
>>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Arun
>>>
>>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <ankitsinghal59@gmail.com
>>> > wrote:
>>>
>>>> Hi Arun,
>>>>
>>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>>> !describe on system.catalog.
>>>>
>>>>
>>>> if not,
>>>> can you share the output of below command. As it seems SYSTEM.CATALOG
>>>> was updated with timestamp greater v4.6 timestamp , and which stopping
>>>> upgrade code to add a new column.
>>>>
>>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Ankit Singhal
>>>>
>>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> After further investigation, we found that Phoenix Upsert query
>>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>>> before 4.6 upgrade?"
>>>>>
>>>>> We do see that clearCache() is being called for 4.7, and 4.7 upgrades
>>>>> from ConnectionQueryServicesImpl class, but not for 4.6
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>>> arun1087@gmail.com> wrote:
>>>>>
>>>>>> James,
>>>>>>
>>>>>> To add more information on this issue, this happens in new phoenix
>>>>>> views associated with brand new tables as well. So, this cannot be an
>>>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>>>> be approached from this point?
>>>>>>
>>>>>> Thanks,
>>>>>> Arun
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
James,
Table SYSTEM.CATALOG is ENABLED
SYSTEM.CATALOG, {TABLE_ATTRIBUTES => {coprocessor$1 =>
'|org.apache.phoenix.coprocessor.ScanRegionObserver|1|', coprocessor$2 =>
'|org.apache
.phoenix.coprocessor.UngroupedAggregateRegionObserver|1|', coprocessor$3 =>
'|org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver|1
|', coprocessor$4 =>
'|org.apache.phoenix.coprocessor.ServerCachingEndpointImpl|1|',
coprocessor$5 => '|org.apache.phoenix.coprocessor.MetaDa
taEndpointImpl|1|', coprocessor$6 =>
'|org.apache.phoenix.coprocessor.MetaDataRegionObserver|2|', METADATA =>
{'SPLIT_POLICY' => 'org.apache.
phoenix.schema.MetaDataSplitPolicy', 'UpgradeTo30' => 'true'}}
COLUMN FAMILIES DESCRIPTION
{NAME => '0', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1000',
MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'true',
BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
1 row in 0.6060 seconds
The above is for describe SYSTEM.CATALOG. The output for scan
'SYSTEM.CATALOG', {RAW=>true} is too huge.
Thanks,
Arun
On Wed, Apr 20, 2016 at 11:19 AM, James Taylor <ja...@apache.org>
wrote:
> Arun,
> Please run the command Ankit mentioned in an HBase shell and post the
> output back here.
> Thanks,
> James
>
>
> On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
> wrote:
>
>> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>>
>> But we do see this column from select statement:
>>
>> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
>> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>>
>>
>>
>> Thanks,
>>
>> Arun
>>
>> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
>> wrote:
>>
>>> Hi Arun,
>>>
>>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>>> !describe on system.catalog.
>>>
>>>
>>> if not,
>>> can you share the output of below command. As it seems SYSTEM.CATALOG
>>> was updated with timestamp greater v4.6 timestamp , and which stopping
>>> upgrade code to add a new column.
>>>
>>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>>
>>>
>>>
>>> Regards,
>>> Ankit Singhal
>>>
>>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com> wrote:
>>>
>>>> After further investigation, we found that Phoenix Upsert query
>>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>>> before 4.6 upgrade?"
>>>>
>>>> We do see that clearCache() is being called for 4.7, and 4.7 upgrades
>>>> from ConnectionQueryServicesImpl class, but not for 4.6
>>>>
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>>> arun1087@gmail.com> wrote:
>>>>
>>>>> James,
>>>>>
>>>>> To add more information on this issue, this happens in new phoenix
>>>>> views associated with brand new tables as well. So, this cannot be an
>>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>>> be approached from this point?
>>>>>
>>>>> Thanks,
>>>>> Arun
>>>>>
>>>>
>>>>
>>>
>>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by James Taylor <ja...@apache.org>.
Arun,
Please run the command Ankit mentioned in an HBase shell and post the
output back here.
Thanks,
James
On Wednesday, April 20, 2016, Arun Kumaran Sabtharishi <ar...@gmail.com>
wrote:
> !describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
>
> But we do see this column from select statement:
>
> select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
> TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
>
>
>
> Thanks,
>
> Arun
>
> On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <ankitsinghal59@gmail.com
> <javascript:_e(%7B%7D,'cvml','ankitsinghal59@gmail.com');>> wrote:
>
>> Hi Arun,
>>
>> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
>> !describe on system.catalog.
>>
>>
>> if not,
>> can you share the output of below command. As it seems SYSTEM.CATALOG was
>> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
>> code to add a new column.
>>
>> scan 'SYSTEM.CATALOG', {RAW=>true}
>>
>>
>>
>> Regards,
>> Ankit Singhal
>>
>> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>>
>> wrote:
>>
>>> After further investigation, we found that Phoenix Upsert query
>>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>>> before 4.6 upgrade?"
>>>
>>> We do see that clearCache() is being called for 4.7, and 4.7 upgrades
>>> from ConnectionQueryServicesImpl class, but not for 4.6
>>>
>>>
>>> Thanks,
>>> Arun
>>>
>>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>>> arun1087@gmail.com <javascript:_e(%7B%7D,'cvml','arun1087@gmail.com');>>
>>> wrote:
>>>
>>>> James,
>>>>
>>>> To add more information on this issue, this happens in new phoenix
>>>> views associated with brand new tables as well. So, this cannot be an
>>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>>> issue yet. Could you throw some ideas on what direction this problem could
>>>> be approached from this point?
>>>>
>>>> Thanks,
>>>> Arun
>>>>
>>>
>>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
!describe SYSTEM.CATALOG is not returning IS_ROW_TIMESTAMP column.
But we do see this column from select statement:
select * from SYSTEM.CATALOG where TABLE_NAME=’TEST_TABLE_1’ AND
TABLE_SCHEM IS NULL AND TENANT_ID IS NULL ;
Thanks,
Arun
On Wed, Apr 20, 2016 at 1:37 AM, Ankit Singhal <an...@gmail.com>
wrote:
> Hi Arun,
>
> Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing
> !describe on system.catalog.
>
>
> if not,
> can you share the output of below command. As it seems SYSTEM.CATALOG was
> updated with timestamp greater v4.6 timestamp , and which stopping upgrade
> code to add a new column.
>
> scan 'SYSTEM.CATALOG', {RAW=>true}
>
>
>
> Regards,
> Ankit Singhal
>
> On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> After further investigation, we found that Phoenix Upsert query
>> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
>> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
>> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
>> before 4.6 upgrade?"
>>
>> We do see that clearCache() is being called for 4.7, and 4.7 upgrades
>> from ConnectionQueryServicesImpl class, but not for 4.6
>>
>>
>> Thanks,
>> Arun
>>
>> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> James,
>>>
>>> To add more information on this issue, this happens in new phoenix views
>>> associated with brand new tables as well. So, this cannot be an
>>> upgrade/migration issue. Not figured out a specific way to reproduce this
>>> issue yet. Could you throw some ideas on what direction this problem could
>>> be approached from this point?
>>>
>>> Thanks,
>>> Arun
>>>
>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Ankit Singhal <an...@gmail.com>.
Hi Arun,
Do you see 'IS_ROW_TIMESTAMP' column in SYSTEM.CATALOG, by doing !describe
on system.catalog.
if not,
can you share the output of below command. As it seems SYSTEM.CATALOG was
updated with timestamp greater v4.6 timestamp , and which stopping upgrade
code to add a new column.
scan 'SYSTEM.CATALOG', {RAW=>true}
Regards,
Ankit Singhal
On Wed, Apr 20, 2016 at 4:25 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> After further investigation, we found that Phoenix Upsert query
> SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
> failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
> this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
> before 4.6 upgrade?"
>
> We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
> ConnectionQueryServicesImpl class, but not for 4.6
>
>
> Thanks,
> Arun
>
> On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> James,
>>
>> To add more information on this issue, this happens in new phoenix views
>> associated with brand new tables as well. So, this cannot be an
>> upgrade/migration issue. Not figured out a specific way to reproduce this
>> issue yet. Could you throw some ideas on what direction this problem could
>> be approached from this point?
>>
>> Thanks,
>> Arun
>>
>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
After further investigation, we found that Phoenix Upsert query
SYSTEM.CATALOG has IS_ROW_TIMESTAMP column, but PTableImpl.getColumn() is
failing with error:"Undefined column. columnName=IS_ROW_TIMESTAMP" . Does
this mean that PTableImpl is reading from cached entity of SYSTEM.CATALOG
before 4.6 upgrade?"
We do see that clearCache() is being called for 4.7, and 4.7 upgrades from
ConnectionQueryServicesImpl class, but not for 4.6
Thanks,
Arun
On Tue, Apr 19, 2016 at 10:22 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> James,
>
> To add more information on this issue, this happens in new phoenix views
> associated with brand new tables as well. So, this cannot be an
> upgrade/migration issue. Not figured out a specific way to reproduce this
> issue yet. Could you throw some ideas on what direction this problem could
> be approached from this point?
>
> Thanks,
> Arun
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
James,
To add more information on this issue, this happens in new phoenix views
associated with brand new tables as well. So, this cannot be an
upgrade/migration issue. Not figured out a specific way to reproduce this
issue yet. Could you throw some ideas on what direction this problem could
be approached from this point?
Thanks,
Arun
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
James,
We cannot afford to follow this workaround, since the names of the phoenix
views that already exist is stored in MySql as metadata, which is used by
our APIs. We still haven't figured out a way to reproduce this issue as
this is only happening in the production environment.
Thanks,
Arun Sabtharishi
On Mon, Apr 18, 2016 at 11:18 AM, James Taylor <ja...@apache.org>
wrote:
> Arun,
>
> If you're blocked by this, here's a potential way of getting around the
> issue you're hitting. This assumes you have all the DDL statements that
> were used to create tables on the cluster:
> - open an HBase shell and disable and drop the SYSTEM.CATALOG
> - open sqlline connected to your HBase cluster (this will cause the
> SYSTEM.CATALOG to be re-created, now empty)
> - open another sqlline specifying a CurrentSCN property on the URL with a
> long value earlier than any existing data for your tables. For example, the
> following would be 1/1/2011:
> ./sqlline.py localhost;CurrentSCN=1293840000000
> - issue each of your CREATE TABLE statements for your tables in this
> sqlline. This will create a Phoenix table and map to the existing HBase
> data that's already there (i.e. you won't lose any data). The setting of
> the CurrentSCN property will prevent Phoenix from adding the empty key
> value to every existing row since we know it's already there (making the
> CREATE TABLE statements run faster and not timeout).
>
> If you could get us more info on how to repro this issue, we can make sure
> it doesn't happen any more as we've never seen this before.
>
> Thanks,
> James
>
> On Mon, Apr 18, 2016 at 9:02 AM, Samarth Jain <sa...@apache.org> wrote:
>
>> Arun,
>>
>> Older phoenix views, created pre-4.6, shouldn't have the ROW_TIMESTAMP
>> column. Was the upgrade done correctly i.e. the server jar upgraded before
>> the client jar? Is it possible to get the complete stack trace? Would be
>> great if you could come up with a test case here to understand better where
>> the problem is happening.
>>
>> On Mon, Apr 18, 2016 at 8:09 AM, Arun Kumaran Sabtharishi <
>> arun1087@gmail.com> wrote:
>>
>>> To add details to the original problem that was mentioned in this email,
>>> we migrated to Phoenix-4.6.1 very recently and this problem started
>>> occurring only after that.
>>>
>>> 1. Checking SYSTEM.CATALOG for some older phoenix views in the same
>>> environment, some of the *phoenix views did not have the
>>> IS_ROW_TIMESTAMP column*. But, all the* newer phoenix views and
>>> only some of the older phoenix views had this column* present. Could
>>> this be possibly because those older phoenix views(which did not have the
>>> column present) did not migrate properly to phoenix 4.6.1?
>>>
>>> 2. Is this column IS_ROW_TIMESTAMP specific to phoenix tables and can
>>> this be accessed from HBase directly? If so, how?
>>>
>>> Thanks,
>>> Arun
>>>
>>
>>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by James Taylor <ja...@apache.org>.
Arun,
If you're blocked by this, here's a potential way of getting around the
issue you're hitting. This assumes you have all the DDL statements that
were used to create tables on the cluster:
- open an HBase shell and disable and drop the SYSTEM.CATALOG
- open sqlline connected to your HBase cluster (this will cause the
SYSTEM.CATALOG to be re-created, now empty)
- open another sqlline specifying a CurrentSCN property on the URL with a
long value earlier than any existing data for your tables. For example, the
following would be 1/1/2011:
./sqlline.py localhost;CurrentSCN=1293840000000
- issue each of your CREATE TABLE statements for your tables in this
sqlline. This will create a Phoenix table and map to the existing HBase
data that's already there (i.e. you won't lose any data). The setting of
the CurrentSCN property will prevent Phoenix from adding the empty key
value to every existing row since we know it's already there (making the
CREATE TABLE statements run faster and not timeout).
If you could get us more info on how to repro this issue, we can make sure
it doesn't happen any more as we've never seen this before.
Thanks,
James
On Mon, Apr 18, 2016 at 9:02 AM, Samarth Jain <sa...@apache.org> wrote:
> Arun,
>
> Older phoenix views, created pre-4.6, shouldn't have the ROW_TIMESTAMP
> column. Was the upgrade done correctly i.e. the server jar upgraded before
> the client jar? Is it possible to get the complete stack trace? Would be
> great if you could come up with a test case here to understand better where
> the problem is happening.
>
> On Mon, Apr 18, 2016 at 8:09 AM, Arun Kumaran Sabtharishi <
> arun1087@gmail.com> wrote:
>
>> To add details to the original problem that was mentioned in this email,
>> we migrated to Phoenix-4.6.1 very recently and this problem started
>> occurring only after that.
>>
>> 1. Checking SYSTEM.CATALOG for some older phoenix views in the same
>> environment, some of the *phoenix views did not have the
>> IS_ROW_TIMESTAMP column*. But, all the* newer phoenix views and
>> only some of the older phoenix views had this column* present. Could
>> this be possibly because those older phoenix views(which did not have the
>> column present) did not migrate properly to phoenix 4.6.1?
>>
>> 2. Is this column IS_ROW_TIMESTAMP specific to phoenix tables and can
>> this be accessed from HBase directly? If so, how?
>>
>> Thanks,
>> Arun
>>
>
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Samarth Jain <sa...@apache.org>.
Arun,
Older phoenix views, created pre-4.6, shouldn't have the ROW_TIMESTAMP
column. Was the upgrade done correctly i.e. the server jar upgraded before
the client jar? Is it possible to get the complete stack trace? Would be
great if you could come up with a test case here to understand better where
the problem is happening.
On Mon, Apr 18, 2016 at 8:09 AM, Arun Kumaran Sabtharishi <
arun1087@gmail.com> wrote:
> To add details to the original problem that was mentioned in this email,
> we migrated to Phoenix-4.6.1 very recently and this problem started
> occurring only after that.
>
> 1. Checking SYSTEM.CATALOG for some older phoenix views in the same
> environment, some of the *phoenix views did not have the IS_ROW_TIMESTAMP
> column*. But, all the* newer phoenix views and only some of the older
> phoenix views had this column* present. Could this be possibly because
> those older phoenix views(which did not have the column present) did not
> migrate properly to phoenix 4.6.1?
>
> 2. Is this column IS_ROW_TIMESTAMP specific to phoenix tables and can this
> be accessed from HBase directly? If so, how?
>
> Thanks,
> Arun
>
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Arun Kumaran Sabtharishi <ar...@gmail.com>.
To add details to the original problem that was mentioned in this email, we
migrated to Phoenix-4.6.1 very recently and this problem started occurring
only after that.
1. Checking SYSTEM.CATALOG for some older phoenix views in the same
environment, some of the *phoenix views did not have the IS_ROW_TIMESTAMP
column*. But, all the* newer phoenix views and only some of the older
phoenix views had this column* present. Could this be possibly because
those older phoenix views(which did not have the column present) did not
migrate properly to phoenix 4.6.1?
2. Is this column IS_ROW_TIMESTAMP specific to phoenix tables and can this
be accessed from HBase directly? If so, how?
Thanks,
Arun
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by "Pindi, Srinivas" <sr...@epsilon.com>.
Yes.
On Apr 13, 2016, at 5:26 PM, Samarth Jain <sa...@gmail.com>> wrote:
Srinivas,
Are you trying to create a phoenix view over an existing HBase table?
On Wed, Apr 13, 2016 at 11:47 AM, Pindi, Srinivas <sr...@epsilon.com>> wrote:
Problem Statement:
While we are trying to create a phoenix view and we are getting the following exception.
When I looked at the source code here is what I found:
if (currentServerSideTableTimeStamp < MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP_4_6_0) {
columnsToAdd += PhoenixDatabaseMetaData.IS_ROW_TIMESTAMP + " " + PBoolean.INSTANCE.getSqlTypeName();
}
Version:
Server phoenix-4.6.1-HBase-1.0-1.17.0.21
Client phoenix-4.6.1-HBase-1.0-1.17.0.21
Error:
Caused by: java.sql.SQLException: Error running CREATE VIEW XYZ ("A" VARCHAR, "B" VARCHAR) AS SELECT * FROM HBASE_TABLE_1 WHERE (KEY >= '0-02UsG-0' AND KEY <= '0-02UsG-g') AND UPDATED_DATE IS NOT NULL
…
…
... 90 more
Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined column. columnName=IS_ROW_TIMESTAMP
at org.apache.phoenix.schema.PTableImpl.getColumn(PTableImpl.java:647)
at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:361)
at org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:386)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:553)
at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:541)
at org.apache.phoenix.jdbc.PhoenixPreparedStatement.getParameterMetaData(PhoenixPreparedStatement.java:246)
at org.apache.phoenix.schema.MetaDataClient.addColumnMutation(MetaDataClient.java:707)
at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1943)
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
Re: Undefined column. columnName=IS_ROW_TIMESTAMP
Posted by Samarth Jain <sa...@gmail.com>.
Srinivas,
Are you trying to create a phoenix view over an existing HBase table?
On Wed, Apr 13, 2016 at 11:47 AM, Pindi, Srinivas <
srinivas.pindi@epsilon.com> wrote:
> *Problem* *Statement*:
>
> While we are trying to create a phoenix view and we are getting the
> following exception.
>
>
>
> When I looked at the source code here is what I found:
>
>
>
> if (currentServerSideTableTimeStamp <
> MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP_4_6_0) {
>
> columnsToAdd += PhoenixDatabaseMetaData.IS_ROW_TIMESTAMP + " " +
> PBoolean.INSTANCE.getSqlTypeName();
>
> }
>
>
>
> *Version*:
>
> Server phoenix-4.6.1-HBase-1.0-1.17.0.21
>
> Client phoenix-4.6.1-HBase-1.0-1.17.0.21
>
>
>
> *Error*:
>
> Caused by: java.sql.SQLException: Error running CREATE VIEW XYZ ("A"
> VARCHAR, "B" VARCHAR) AS SELECT * FROM HBASE_TABLE_1 WHERE (KEY >=
> '0-02UsG-0' AND KEY <= '0-02UsG-g') AND UPDATED_DATE IS NOT NULL
>
> …
>
> …
>
> ... 90 more
>
> Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504
> (42703): Undefined column. columnName=IS_ROW_TIMESTAMP
>
> at
> org.apache.phoenix.schema.PTableImpl.getColumn(PTableImpl.java:647)
>
> at
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:361)
>
> at
> org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:386)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:553)
>
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:541)
>
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.getParameterMetaData(PhoenixPreparedStatement.java:246)
>
> at
> org.apache.phoenix.schema.MetaDataClient.addColumnMutation(MetaDataClient.java:707)
>
> at
> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1943)
>
>
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>