[jira] [Work logged] (BEAM-9946) Enhance Partition transform to provide partitionfn with SideInputs

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9946?focusedWorklogId=433351&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433351 ]

ASF GitHub Bot logged work on BEAM-9946:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 14/May/20 20:52
            Start Date: 14/May/20 20:52
    Worklog Time Spent: 10m 
      Work Description: aaltay commented on pull request #11682:
URL: https://github.com/apache/beam/pull/11682#issuecomment-628880635


   This LGTM and well written. I will trigger the tests.
   
   Added @kennknowles in case I am missing something related to the java apis.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 433351)
    Remaining Estimate: 95h 50m  (was: 96h)
            Time Spent: 10m

> Enhance Partition transform to provide partitionfn with SideInputs
> ------------------------------------------------------------------
>
>                 Key: BEAM-9946
>                 URL: https://issues.apache.org/jira/browse/BEAM-9946
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core
>            Reporter: Darshan Jani
>            Assignee: Darshan Jani
>            Priority: Major
>   Original Estimate: 96h
>          Time Spent: 10m
>  Remaining Estimate: 95h 50m
>
> Currently _Partition_ transform can partition a collection into n collections based on only _element_ value in _PartitionFn_ to decide on which partition a particular element belongs to.
> {code:java}
> public interface PartitionFn<T> extends Serializable {
>     int partitionFor(T elem, int numPartitions);
>   }
> public static <T> Partition<T> of(int numPartitions, PartitionFn<? super T> partitionFn) {
>     return new Partition<>(new PartitionDoFn<T>(numPartitions, partitionFn));
>   }
> {code}
> It will be useful to introduce new API with additional _sideInputs_ provided to partition function. User will be able to write logic to use both _element_ value and _sideInputs_ to decide on which partition a particular element belongs to.
> Option-1: Proposed new API:
> {code:java}
>   public interface PartitionWithSideInputsFn<T> extends Serializable {
>     int partitionFor(T elem, int numPartitions, Context c);
>   }
> public static <T> Partition<T> of(int numPartitions, PartitionWithSideInputsFn<? super T> partitionFn, Requirements requirements) {
>  ...
>   }
> {code}
> User can use any of the two APIs as per there partitioning function logic.
> Option-2: Redesign old API with Builder Pattern which can provide optionally a _Requirements_ with _sideInputs._ Deprecate old API.
> {code:java}
> // using sideviews
> Partition.into(numberOfPartitions).via(
> fn(
>   (input,c) ->  {
>     // use c.sideInput(view)
>     // use input
>     // return partitionnumber
>  },requiresSideInputs(view))
> )
> // without using sideviews
> Partition.into(numberOfPartitions).via(
> fn((input,c) ->  {
>     // use input
>     // return partitionnumber
>  })
> )
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)