You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/06/23 22:38:21 UTC
[GitHub] [hive] vihangk1 opened a new pull request #2425: HIVE-25281: Add optional fields to enable returning filemetadata
vihangk1 opened a new pull request #2425:
URL: https://github.com/apache/hive/pull/2425
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
3. Ensure you have added or run the appropriate tests for your PR:
4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX: Your PR title ...'.
5. Be sure to keep the PR description updated to reflect all changes.
6. Please write your PR title to summarize what this PR proposes.
7. If possible, provide a concise example to reproduce the issue for a faster review.
-->
### What changes were proposed in this pull request?
This PR introduces new fields in Table and Partition thrift structs. These fields are optional and currently HMS does not set these fields by default. However, such fields are useful for non-hive SQL engines to integrate their metadata cache over with HMS API. For example, see IMPALA-10612.
### Why are the changes needed?
See description above.
### Does this PR introduce _any_ user-facing change?
It introduces new optional fields to the Table and Partition objects. These should not impact any current installations for HMS clients.
### How was this patch tested?
From hive's perspective the testing was manual. In order to test this functionality, we would need to have some component which has ability to send the filemetdata over HMS API. This is not available currently in Hive but there were tests introduced in IMPALA-10612 to test the basic end-to-end functionality of this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 merged pull request #2425: HIVE-25281: Add optional fields to enable returning filemetadata
Posted by GitBox <gi...@apache.org>.
vihangk1 merged pull request #2425:
URL: https://github.com/apache/hive/pull/2425
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kishendas commented on a change in pull request #2425: HIVE-25281: Add optional fields to enable returning filemetadata
Posted by GitBox <gi...@apache.org>.
kishendas commented on a change in pull request #2425:
URL: https://github.com/apache/hive/pull/2425#discussion_r673413945
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -610,6 +637,8 @@ struct Partition {
10: optional i64 writeId=-1,
11: optional bool isStatsCompliant,
12: optional ColumnStatistics colStats // column statistics for partition
Review comment:
Comma is missing after colStats. Does this still work fine, when you generate the code ?
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -610,6 +637,8 @@ struct Partition {
10: optional i64 writeId=-1,
11: optional bool isStatsCompliant,
12: optional ColumnStatistics colStats // column statistics for partition
+ 13: optional FileMetadata fileMetadata // optional serialized file-metadata useful
+ // for certain execution engines
Review comment:
Please align the comment
##########
File path: standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift
##########
@@ -595,6 +619,9 @@ struct Table {
24: optional list<string> requiredWriteCapabilities
25: optional i64 id, // id of the table. It will be ignored if set. It's only for
// read purposed
+ 26: optional FileMetadata fileMetadata // optional serialized file-metadata for this table
+ // for certain execution engines
Review comment:
You want to align this comment with previous line, so that its easy to read ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] vihangk1 commented on pull request #2425: HIVE-25281: Add optional fields to enable returning filemetadata
Posted by GitBox <gi...@apache.org>.
vihangk1 commented on pull request #2425:
URL: https://github.com/apache/hive/pull/2425#issuecomment-883765338
There is no code changes in the latest commit (only some comment realignments). I am going to merge the patch without waiting for the precommit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org