[GitHub] [doris] Tanya-W opened a new issue, #19061: [Enhancement] Improve alter inverted index performance with light weight add or drop inverted index

["Tanya-W (via GitHub)" <gi...@apache.org>]

Tanya-W opened a new issue, #19061:
URL: https://github.com/apache/doris/issues/19061

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Description
   
   In the current use, add or drop inverted index by directly  schema change, which convert base tablet data and rewrite new tablet, it's a heavy work. Because of the implementation inverted index, there is no need to read or write irrelevant data, the performance can be improved greatly.
   
   ### Solution
   
   In terms of usage, for `add index`, there is a significant difference with the previous, `add index` only effective for new data, we have added the `build index` syntax, which is mainly used to complete the inverted index for stock data.
   
   - `build index` syntax:
   ```
   BUILD INDEX index_name ON table_name;
   BUILD INDEX index_name ON table_name PARTITION(p1, p2);
   ```
   (`add index` must have been executed before `build index`)
   
   Based on the above, `add index` first modify fe's meta and then immediately return to the user, the read and load need with the latest schema of fe, for newly load, the inverted index file can be synchronously generated.
   
   Similarly, `drop index` first modify the meta of fe and then immediately return to the user, the read and load need with the latest schema of fe, will asynchronous delete the inverted index files.
   
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] dataroaring closed issue #19061: [Enhancement] Improve alter inverted index performance with light weight add or drop inverted index

["dataroaring (via GitHub)" <gi...@apache.org>]

dataroaring closed issue #19061: [Enhancement] Improve alter inverted index performance with light weight add or drop inverted index
URL: https://github.com/apache/doris/issues/19061


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org