[jira] [Updated] (ATLAS-5227) Support disabling composite index

["Pinal Shah (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pinal Shah updated ATLAS-5227:
------------------------------
    Description: 
+*Description:*+

This Jira is to support disabling the active index in janusgraph.
For Example : For ddl.queryText , isIndexable=true.
*This patch handles the support to make any attribute to isIndexable=false.*

 

+*Improvement to Disable unnecessary Indexes:*+
 # *Write Latency* will drop significantly because JanusGraph no longer has to create a secondary index entry for that massive string.
 # *HBase Stability* will improve because we might stop hitting "Key Size Limit" exceptions.
 # *Storage Savings:* It will not create indexes

 

+*Limitation:*+
 - graph search corresponding to this index will not be possible

 

+*Explanation:*+

To properly remove an index, we need to follow below updated in sequence
 # *DISABLE_INDEX:* Sets state to {{{}DISABLED{}}}. The index is no longer used for queries and stops receiving updates.
 # *DISCARD_INDEX:* Removes all data from the index storage. The index state becomes {{{}DISCARDED{}}}.
 # *DROP_INDEX:* Removes the index definition from the JanusGraph schema entirely.

But performing *DISCARD_INDEX* and *DROP_INDEX actions* can be too expensive when there is huge indexed data. Hence this patch only handles  *DISABLE_INDEX*

 

+*Acceptance Criteria:*+ 
 - User functionality should remain unchanged
 - ATLAS-5228 should be resolved 
 - Atlas startup time should be comparable
 - Once index is set to false, we cannot enable it

 

+*Testing Done:*+

For Postgres and Hbase backend
 * on fresh db checked ddl.queryText is not getting indexed
 * For existing data, index is changed and exception is not seen
 * compared entity (having ddl.queryText) creation time, with and without indexing
 * compared Atlas startup time having historical data with the fix and without fix

For Postgres backend
 * exception row big size is not seen (ATLAS-5228)

  was:
+*Description:*+

This Jira is to support disabling the active index in janusgraph.
For Example : For ddl.queryText , isIndexable=true.
*This patch handles the support to make any attribute to isIndexable=false.*

 

+*Improvement to Disable unnecessary Indexes:*+
 # *Write Latency* will drop significantly because JanusGraph no longer has to create a secondary index entry for that massive string.
 # *HBase Stability* will improve because we might stop hitting "Key Size Limit" exceptions.
 # *Storage Savings:* It will not create indexes

 

+*Limitation:*+
 - graph search corresponding to this index will not be possible

 

+*Explanation:*+

To properly remove an index, we need to follow below updated in sequence
 # *DISABLE_INDEX:* Sets state to {{{}DISABLED{}}}. The index is no longer used for queries and stops receiving updates.
 # *DISCARD_INDEX:* Removes all data from the index storage. The index state becomes {{{}DISCARDED{}}}.
 # *DROP_INDEX:* Removes the index definition from the JanusGraph schema entirely.

But performing *DISCARD_INDEX* and *DROP_INDEX actions* can be too expensive when there is huge indexed data. Hence this patch only handles  *DISABLE_INDEX*

 

+*Acceptance Criteria:*+ 
 - User functionality should remain unchanged
 - ATLAS-5228 should be resolved 
 - Atlas startup time should be comparable
 - Once index is set to false, we cannot enable it


> Support disabling composite index
> ---------------------------------
>
>                 Key: ATLAS-5227
>                 URL: https://issues.apache.org/jira/browse/ATLAS-5227
>             Project: Atlas
>          Issue Type: Task
>            Reporter: Pinal Shah
>            Assignee: Pinal Shah
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> +*Description:*+
> This Jira is to support disabling the active index in janusgraph.
> For Example : For ddl.queryText , isIndexable=true.
> *This patch handles the support to make any attribute to isIndexable=false.*
>  
> +*Improvement to Disable unnecessary Indexes:*+
>  # *Write Latency* will drop significantly because JanusGraph no longer has to create a secondary index entry for that massive string.
>  # *HBase Stability* will improve because we might stop hitting "Key Size Limit" exceptions.
>  # *Storage Savings:* It will not create indexes
>  
> +*Limitation:*+
>  - graph search corresponding to this index will not be possible
>  
> +*Explanation:*+
> To properly remove an index, we need to follow below updated in sequence
>  # *DISABLE_INDEX:* Sets state to {{{}DISABLED{}}}. The index is no longer used for queries and stops receiving updates.
>  # *DISCARD_INDEX:* Removes all data from the index storage. The index state becomes {{{}DISCARDED{}}}.
>  # *DROP_INDEX:* Removes the index definition from the JanusGraph schema entirely.
> But performing *DISCARD_INDEX* and *DROP_INDEX actions* can be too expensive when there is huge indexed data. Hence this patch only handles  *DISABLE_INDEX*
>  
> +*Acceptance Criteria:*+ 
>  - User functionality should remain unchanged
>  - ATLAS-5228 should be resolved 
>  - Atlas startup time should be comparable
>  - Once index is set to false, we cannot enable it
>  
> +*Testing Done:*+
> For Postgres and Hbase backend
>  * on fresh db checked ddl.queryText is not getting indexed
>  * For existing data, index is changed and exception is not seen
>  * compared entity (having ddl.queryText) creation time, with and without indexing
>  * compared Atlas startup time having historical data with the fix and without fix
> For Postgres backend
>  * exception row big size is not seen (ATLAS-5228)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)