Re: [I] Being able to write SQL against the metadata table [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #16078:
URL: https://github.com/apache/hudi/issues/16078#issuecomment-4795528495

   This appears to be addressed on current `master`. SQL queries against the metadata table are supported via:
   
   **1. The `hudi_metadata()` table-valued function** (added in [HUDI-7294], commit `108a885b4db6`):
   ```sql
   SELECT * FROM hudi_metadata('/path/to/table');
   SELECT * FROM hudi_metadata('db.table_name');
   SELECT key FROM hudi_metadata('table') WHERE type = 7;
   ```
   See `HoodieMetadataTableValuedFunction.scala` and tests in `TestHoodieTableValuedFunction.scala`.
   
   **2. Direct path-based reads** of the underlying MOR metadata table:
   ```scala
   spark.read.format("hudi").load("/path/to/table/.hoodie/metadata")
   spark.read.format("hudi").load("/path/to/table/.hoodie/metadata/files")
   spark.read.format("hudi").load("/path/to/table/.hoodie/metadata/column_stats")
   ```
   Exercised in `TestMetadataTableWithSparkDataSource`.
   
   **3. Inspection procedures** like `CALL show_metadata_table_stats(...)` and `CALL show_metadata_table_files(...)`.
   
   If there is a specific SQL access pattern still missing here (e.g. `tableName.metadata` reference syntax), please reopen with the concrete query you'd like to run; otherwise this can be closed as resolved by HUDI-7294.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org