You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/04/27 10:41:13 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue #210: Add support for group by Decimal numbers
alamb opened a new issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210
In the context of https://github.com/apache/arrow-datafusion/issues/107 from @joshuataylor
This will likely require https://github.com/apache/arrow-rs/issues/230 (support for pretty-printing decimal numbers)
**Describe the solution you'd like**
It should be possible to group by data in columns of `DataType::Decimal` type
Right now you get an error such as:
```
(Internal("Unsupported GROUP BY type creating key Decimal(9, 0)"))
```
** Reproducer **
In the datafusion-cli:
```
CREATE EXTERNAL TABLE something STORED AS PARQUET LOCATION 'demo.parquet';
select O_ORDERKEY from something group by O_ORDERKEY;
```
Where demo.parquet is here: https://drive.google.com/file/d/1aCW7SW2rUVioSePduhgo_91F5-xDMyjp/view?usp=sharing
(note the file is large, so I am not sure how long this query will take)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210#issuecomment-1023658446
Thanks for all your help @liukun4515 and pushing it through. Really nicely done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210#issuecomment-1022877445
thanks for @alamb and @houqp pushing this feature forward.
There are many tasks about decimal or data type to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210#issuecomment-1022804688
@alamb
this issue can be closed
```
❯ \d food
+---------------+--------------+------------+-------------+-----------------+-------------+
| table_catalog | table_schema | table_name | column_name | data_type | is_nullable |
+---------------+--------------+------------+-------------+-----------------+-------------+
| datafusion | public | food | a | Decimal(10, 5) | NO |
| datafusion | public | food | b | Decimal(20, 15) | NO |
| datafusion | public | food | c | Boolean | NO |
+---------------+--------------+------------+-------------+-----------------+-------------+
3 rows in set. Query took 0.010 seconds.
❯ select count(*),a from food group by a;
+-----------------+---------+
| COUNT(UInt8(1)) | a |
+-----------------+---------+
| 3 | 0.00003 |
| 1 | 0.00001 |
| 4 | 0.00004 |
| 5 | 0.00005 |
| 2 | 0.00002 |
+-----------------+---------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] houqp closed issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
houqp closed issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] houqp commented on issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
houqp commented on issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210#issuecomment-1022871940
good work @liukun4515 :tada:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 edited a comment on issue #210: Add support for group by Decimal numbers
Posted by GitBox <gi...@apache.org>.
liukun4515 edited a comment on issue #210:
URL: https://github.com/apache/arrow-datafusion/issues/210#issuecomment-1022877445
thanks for @alamb and @houqp pushing this feature forward.
There are many tasks about decimal or data type to do.
For example:
1. move some operation or logic to arrow-rs、kernel
2. improve the performance of decimal operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org