You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/04/13 20:13:00 UTC

[jira] [Work logged] (BEAM-9748) Move Reparallelize transform to Reshuffle

     [ https://issues.apache.org/jira/browse/BEAM-9748?focusedWorklogId=421606&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-421606 ]

ASF GitHub Bot logged work on BEAM-9748:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 13/Apr/20 20:12
            Start Date: 13/Apr/20 20:12
    Worklog Time Spent: 10m 
      Work Description: iemejia commented on pull request #11406: [BEAM-9748] Move Reparallelize transform to Reshuffle
URL: https://github.com/apache/beam/pull/11406
 
 
   Some DoFn based IOs like JdbcIO and RedisIO rely on the Reparallelize transform, a combination of a an empty PCollectionView and Reshuffle to force the materialization and reparallelize a PCollection. The idea of this issue is to extract this transform and expose it as part of the internal Reshuffle transform to avoid repeating the code for transforms (notably IOs) that require to reparallelize its output.
   
   R: @lukecwik 
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 421606)
    Remaining Estimate: 0h
            Time Spent: 10m

> Move Reparallelize transform to Reshuffle
> -----------------------------------------
>
>                 Key: BEAM-9748
>                 URL: https://issues.apache.org/jira/browse/BEAM-9748
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-java-core
>            Reporter: Ismaël Mejía
>            Assignee: Ismaël Mejía
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Some DoFn based IOs like JdbcIO and RedisIO rely on the Reparallelize transform,
> a combination of a an empty PCollectionView and Reshuffle to force the
> materialization and reparallelize a PCollection. The idea of this issue is to
> extract this transform and expose it as part of the internal Reshuffle
> transform to avoid repeating the code for transforms (notably IOs) that require
> to reparallelize its output.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)