You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Ji Xu (JIRA)" <ji...@apache.org> on 2018/12/06 07:30:00 UTC

[jira] [Updated] (ARROW-3947) [Python] query distinct values of a given partition from a ParquetDataset

     [ https://issues.apache.org/jira/browse/ARROW-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ji Xu updated ARROW-3947:
-------------------------
    Summary: [Python] query distinct values of a given partition from a ParquetDataset  (was: query distinct values of a given partition from a ParquetDataset)

> [Python] query distinct values of a given partition from a ParquetDataset
> -------------------------------------------------------------------------
>
>                 Key: ARROW-3947
>                 URL: https://issues.apache.org/jira/browse/ARROW-3947
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>    Affects Versions: 0.10.0
>         Environment: MacOSX, Python 3.6, 
>            Reporter: Ji Xu
>            Priority: Minor
>
> Right now the values of a given partition from a `ParquetDataset` is buried inside `ParquetDataset.pieces`, a bit inconvenient for the user to dig out this information. A helper function/method to perform this task in `ParquetDataset` class would be very helpful for the users.
> A pure personal opinion on the name of this method: `ParquetDataset.select_distinct()` with partition_name as the positional arg, to resemble SQL `SELECT DISTINCT column FROM table`.
> I'm not sure how to contribute here on Jira, so I created this [GitHub Gist |https://gist.github.com/xujiboy/c3fcc47f720ed9adf2260c5d0ba8aed2]as an possible solution for this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)