You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "lokeshj1703 (via GitHub)" <gi...@apache.org> on 2024/12/03 10:58:23 UTC
[PR] [DOCS] Add indexing examples and info for 1.0 [hudi]
lokeshj1703 opened a new pull request, #12409:
URL: https://github.com/apache/hudi/pull/12409
### Change Logs
Adds documentation for indexing features in 1.0.
### Impact
Adds documentation for indexing features in 1.0.
### Risk level (write none, low medium or high below)
low
### Documentation Update
NA
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DOCS] Add indexing examples and info for 1.0 [hudi]
Posted by "vinothchandar (via GitHub)" <gi...@apache.org>.
vinothchandar merged PR #12409:
URL: https://github.com/apache/hudi/pull/12409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DOCS] Add indexing examples and info for 1.0 [hudi]
Posted by "lokeshj1703 (via GitHub)" <gi...@apache.org>.
lokeshj1703 commented on code in PR #12409:
URL: https://github.com/apache/hudi/pull/12409#discussion_r1867641088
##########
website/docs/metadata_indexing.md:
##########
@@ -10,18 +10,65 @@ The [pluggable indexing subsystem](https://www.onehouse.ai/blog/introducing-mult
of Hudi depends on the metadata table. Different types of index, from `files` index for locating records efficiently
to `column_stats` index for data skipping, are part of the metadata table. A fundamental tradeoff in any data system
that supports indices is to balance the write throughput with index updates. A brute-force way is to lock out the writes
-while indexing. However, very large tables can take hours to index. This is where Hudi's novel asynchronous metadata
-indexing comes into play.
+while indexing. Hudi supports index creation using SQL, Datasource as well as async indexing. However, very large tables
Review Comment:
Addressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DOCS] Add indexing examples and info for 1.0 [hudi]
Posted by "vinothchandar (via GitHub)" <gi...@apache.org>.
vinothchandar commented on code in PR #12409:
URL: https://github.com/apache/hudi/pull/12409#discussion_r1867544457
##########
website/docs/metadata_indexing.md:
##########
@@ -10,18 +10,65 @@ The [pluggable indexing subsystem](https://www.onehouse.ai/blog/introducing-mult
of Hudi depends on the metadata table. Different types of index, from `files` index for locating records efficiently
to `column_stats` index for data skipping, are part of the metadata table. A fundamental tradeoff in any data system
that supports indices is to balance the write throughput with index updates. A brute-force way is to lock out the writes
-while indexing. However, very large tables can take hours to index. This is where Hudi's novel asynchronous metadata
-indexing comes into play.
+while indexing. Hudi supports index creation using SQL, Datasource as well as async indexing. However, very large tables
Review Comment:
please cover how concurrency control is done.. Just write out index building protocols as simple bullets.
##########
website/docs/sql_ddl.md:
##########
@@ -582,6 +597,28 @@ SHOW PARTITIONS hudi_table;
--Drop partition：
ALTER TABLE hudi_table DROP PARTITION (dt='2021-12-09', hh='10');
```
+
+### Show and drop index
+
+**Syntax**
+
+```sql
+-- Show Indexes
+SHOW INDEXES FROM tableIdentifier;
Review Comment:
please use consistent case (upper, lower, camel) as rest of docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DOCS] Add indexing examples and info for 1.0 [hudi]
Posted by "lokeshj1703 (via GitHub)" <gi...@apache.org>.
lokeshj1703 commented on code in PR #12409:
URL: https://github.com/apache/hudi/pull/12409#discussion_r1867572371
##########
website/docs/sql_ddl.md:
##########
@@ -582,6 +597,28 @@ SHOW PARTITIONS hudi_table;
--Drop partition：
ALTER TABLE hudi_table DROP PARTITION (dt='2021-12-09', hh='10');
```
+
+### Show and drop index
+
+**Syntax**
+
+```sql
+-- Show Indexes
+SHOW INDEXES FROM tableIdentifier;
Review Comment:
Addressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org