You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Francois Saint-Jacques (Jira)" <ji...@apache.org> on 2020/04/15 19:21:00 UTC
[jira] [Commented] (ARROW-8276) [C++][Dataset] Scanning a Fragment
does not take into account the partition columns
[ https://issues.apache.org/jira/browse/ARROW-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084327#comment-17084327 ]
Francois Saint-Jacques commented on ARROW-8276:
-----------------------------------------------
[~jorisvandenbossche] I'm refactoring this part with ARROW-8065, I don't think `get_fragments` should do any column projection. It should just return fragments from a dataset. On the other side, `dataset.scan` and `dataset.to_table` should do said transformation.
> [C++][Dataset] Scanning a Fragment does not take into account the partition columns
> -----------------------------------------------------------------------------------
>
> Key: ARROW-8276
> URL: https://issues.apache.org/jira/browse/ARROW-8276
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Joris Van den Bossche
> Assignee: Ben Kietzman
> Priority: Major
> Labels: dataset, pull-request-available
> Fix For: 0.17.0
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Follow-up on ARROW-8061, the {{to_table}} method doesn't work for fragments created from a partitioned dataset.
> (will add a reproducer later)
> cc [~bkietz]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)