You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2023/01/15 09:40:34 UTC
[GitHub] [arrow-datafusion] xiaoyong-z opened a new issue, #4913: Support describe FILE in datafusion-cli
xiaoyong-z opened a new issue, #4913:
URL: https://github.com/apache/arrow-datafusion/issues/4913
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
Currently, dafausion support 'create external table xxx stored as xxx location FILE and select * from FILE,
to know the columns/types in a file, users has to create an external table, and describe the table.
Also, sometimes the infer type is wrong, to make it right, user need to drop the table, and recreate a table
with the specify schema.
DuckDB supports using the following sql to get the infer type from a parquet file
DESCRIBE SELECT * FROM 'test.parquet';
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
Add support for DESCRIBE 'test.parquet' in datafusion;
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #4913: Support Describe FILE in datafusion-cli
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #4913:
URL: https://github.com/apache/arrow-datafusion/issues/4913#issuecomment-1383142122
I bet you could implement this fairly simply by special casing such a descirbe prediate (I think now it runs a query against the schema information table0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] xiaoyong-z commented on issue #4913: Support describe FILE in datafusion-cli
Posted by GitBox <gi...@apache.org>.
xiaoyong-z commented on issue #4913:
URL: https://github.com/apache/arrow-datafusion/issues/4913#issuecomment-1383104017
Is this feature planned? @alamb
If no one works on this, i will contribute to this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #4913: Support Describe FILE in datafusion-cli
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #4913:
URL: https://github.com/apache/arrow-datafusion/issues/4913#issuecomment-1383141864
Hi @xiaoyong-z -- I don't know of anyone working on this
Note datafusion already supports `describe` and selecting from parquet files (in datafusion-cli -- https://github.com/apache/arrow-datafusion/issues/4850 tracks supporting it more generally and I think that @matthewwillian said he was looking into)
I actually would have expected `describe <foo.parquet>` to work as it should simply be working using the existing table provider machinery. However, when I tried it:
```shell
cargo run datafusion-cli
```
It did not:
```sql
❯ describe '/Users/alamb/Software/duckdb-polr/tools/juliapkg/data/invoice.parquet';
0 rows in set. Query took 0.002 seconds.
❯ select * from '/Users/alamb/Software/duckdb-polr/tools/juliapkg/data/invoice.parquet' limit 10;
+-----------+------------+-----------------------------------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
| InvoiceId | CustomerId | InvoiceDate | BillingAddress | BillingCity | BillingState | BillingCountry | BillingPostalCode | Total |
+-----------+------------+-----------------------------------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
| 1 | 2 | 2007-01-01T00:00:00 (Unknown Time Zone 'UTC') | Theodor-Heuss-Straße 34 | Stuttgart | | Germany | 70174 | 1.98 |
| 2 | 4 | 2007-01-02T00:00:00 (Unknown Time Zone 'UTC') | Ullevålsveien 14 | Oslo | | Norway | 0171 | 3.95 |
| 3 | 8 | 2007-01-03T00:00:00 (Unknown Time Zone 'UTC') | Grétrystraat 63 | Brussels | | Belgium | 1000 | 5.94 |
| 4 | 14 | 2007-01-06T00:00:00 (Unknown Time Zone 'UTC') | 8210 111 ST NW | Edmonton | AB | Canada | T6G 2C7 | 8.91 |
| 5 | 23 | 2007-01-11T00:00:00 (Unknown Time Zone 'UTC') | 69 Salem Street | Boston | MA | USA | 2113 | 13.85 |
| 6 | 37 | 2007-01-19T00:00:00 (Unknown Time Zone 'UTC') | Berger Straße 10 | Frankfurt | | Germany | 60316 | 0.99 |
| 7 | 38 | 2007-02-01T00:00:00 (Unknown Time Zone 'UTC') | Barbarossastraße 19 | Berlin | | Germany | 10779 | 1.98 |
| 8 | 40 | 2007-02-01T00:00:00 (Unknown Time Zone 'UTC') | 8, Rue Hanovre | Paris | | France | 75002 | 1.98 |
| 9 | 42 | 2007-02-02T00:00:00 (Unknown Time Zone 'UTC') | 9, Place Louis Barthou | Bordeaux | | France | 33000 | 3.95 |
| 10 | 46 | 2007-02-03T00:00:00 (Unknown Time Zone 'UTC') | 3 Chatham Street | Dublin | Dublin | Ireland | | 5.94 |
+-----------+------------+-----------------------------------------------+-------------------------+-------------+--------------+----------------+-------------------+-------+
10 rows in set. Query took 0.003 seconds.
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #4913: Support Describe FILE in datafusion-cli
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #4913: Support Describe FILE in datafusion-cli
URL: https://github.com/apache/arrow-datafusion/issues/4913
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org