[Bug] SI Compatibility Issue

[Indhumathi <in...@gmail.com>]

Hello all,

Currently, for Secondary Index metadata, we are adding the below two
properties 
to index  table and Fact table on SI creation.
1. indextableexists -> will be set to true, if table has SI. will be null
for SI table.
2. isIndexTable -> to know if the table is SI or not

When CarbonInternalMetastore#refreshIndexInfo is called on lookUpRelation, 
we update the above properties to table, based on the
indexMetadata information stored in Hive.

Issue scenario:
For compatibility scenario, where fact table and SI are created in older
versions and 
if we need to upgrade those tables to latest version, we execute the below
commands.
1. Refresh table -> register the table with metaStore
2. Register Index command
    -> loads index information to Hive and set index formation to Fact
table.

Currently, after refresh index table and before register index, we can
perform any 
operation on index Table(including load,alter, iud,..),  because
isIndexTable property 
will be set only during Register index. These operations will affect
Registering the 
index to fact table.

Do we need to block those operations, even before register index Or can  we 
allow it and check Fact and SI table Sync during register index?

Please let me know your solutions about the same.

Regards,
Indhumathi M






--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Bug] SI Compatibility Issue

[akashrn5 <ak...@gmail.com>]

Hi,

I think we cant block any operations on table just for this reason. Since we
have give two commands for it, we cant block user. 

1. Either we need to handle all these during refresh table only instead of
having a one more register index command which will solve the issue.
2. Or we need handle from the code based on the update to the property
happened during any operation on table after refresh and before register.

Regards,
Akash R 



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/