You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/10/27 21:02:00 UTC

[jira] [Updated] (BEAM-13133) sample() imposes partitioning by index unnecessarily

     [ https://issues.apache.org/jira/browse/BEAM-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-13133:
-----------------------------------
    Status: Open  (was: Triage Needed)

> sample() imposes partitioning by index unnecessarily
> ----------------------------------------------------
>
>                 Key: BEAM-13133
>                 URL: https://issues.apache.org/jira/browse/BEAM-13133
>             Project: Beam
>          Issue Type: Task
>          Components: dsl-dataframe
>            Reporter: Brian Hulette
>            Assignee: Brian Hulette
>            Priority: P2
>
> I noticed that sample() requires data to repartitioned when it's used at the beginning of a series of dataframe commands. In practice we should be able to sample within arbitrary partitions before combining the partitions to produce the final result.
> It looks like the root cause is that our sample expressions require partitioning by index, rather than arbitrary partitioning.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)