You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/04/26 18:53:59 UTC
[GitHub] [arrow-datafusion] returnString opened a new pull request #205: Allow table providers to indicate their type for catalog metadata
returnString opened a new pull request #205:
URL: https://github.com/apache/arrow-datafusion/pull/205
# Which issue does this PR close?
Closes #191.
# Rationale for this change
This allows registered TableProviders to approximately indicate the mechanism used to retrieve data during query execution, which in turn allows tables to be logged with a `table_type` column in `information_schema.tables`. This was previously partially supported by classifying all registered tables as `BASE TABLE` and all generated system tables as `VIEW`.
This means that DataFusion consumers can now build systems that allow end-users to differentiate between relation types. For example, you could build a TableProvider that stores a query plan, use this as a view in your execution context, and end users would be aware that any queries over it in turn trigger use a query internally.
# What changes are included in this PR?
- Added `TableType` enum
- Exposed new trait method, `TableProvider::table_type`
- Updated information_schema builder to use new table types
# Are there any user-facing changes?
`TableProvider::table_type` is introduced as a new trait method, but comes with a default implementation returning `TableType::Base` to preserve back-compact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #205: Allow table providers to indicate their type for catalog metadata
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #205:
URL: https://github.com/apache/arrow-datafusion/pull/205#issuecomment-827080255
# [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#205](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (08e0044) into [master](https://codecov.io/gh/apache/arrow-datafusion/commit/ddaea81f9f46e918b5ab4e6257f1963b2a8a0f15?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ddaea81) will **decrease** coverage by `0.01%`.
> The diff coverage is `72.22%`.
> :exclamation: Current head 08e0044 differs from pull request most recent head 589d8fa. Consider uploading reports for the commit 589d8fa to get more accurate results
[](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #205 +/- ##
==========================================
- Coverage 76.35% 76.33% -0.02%
==========================================
Files 134 134
Lines 23018 23042 +24
==========================================
+ Hits 17576 17590 +14
- Misses 5442 5452 +10
```
| [Impacted Files](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [datafusion/src/execution/context.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/205/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9zcmMvZXhlY3V0aW9uL2NvbnRleHQucnM=) | `92.63% <68.75%> (-0.31%)` | :arrow_down: |
| [datafusion/src/catalog/information\_schema.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/205/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9zcmMvY2F0YWxvZy9pbmZvcm1hdGlvbl9zY2hlbWEucnM=) | `86.72% <72.22%> (-2.06%)` | :arrow_down: |
| [datafusion/src/datasource/datasource.rs](https://codecov.io/gh/apache/arrow-datafusion/pull/205/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZGF0YWZ1c2lvbi9zcmMvZGF0YXNvdXJjZS9kYXRhc291cmNlLnJz) | `100.00% <100.00%> (ø)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [ddaea81...589d8fa](https://codecov.io/gh/apache/arrow-datafusion/pull/205?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a change in pull request #205: Allow table providers to indicate their type for catalog metadata
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #205:
URL: https://github.com/apache/arrow-datafusion/pull/205#discussion_r620669295
##########
File path: datafusion/src/datasource/datasource.rs
##########
@@ -66,6 +66,17 @@ pub enum TableProviderFilterPushDown {
Exact,
}
+/// Indicates the type of this table for metadata/catalog purposes.
+#[derive(Debug, Clone, Copy)]
+pub enum TableType {
+ /// An ordinary physical table.
+ Base,
+ /// A non-materialised table that itself uses a query internally to provide data.
+ View,
Review comment:
A preview of things to come, perhaps :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #205: Allow table providers to indicate their type for catalog metadata
Posted by GitBox <gi...@apache.org>.
alamb merged pull request #205:
URL: https://github.com/apache/arrow-datafusion/pull/205
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org