You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Darshan Jani (Jira)" <ji...@apache.org> on 2020/05/02 16:48:00 UTC
[jira] [Commented] (BEAM-9825) Transforms for Intersect, Difference
and Commons
[ https://issues.apache.org/jira/browse/BEAM-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098020#comment-17098020 ]
Darshan Jani commented on BEAM-9825:
------------------------------------
Thanks [~amaliujia].
Makes sense to follow your advices.
I am thinking to implement all of these transform can work on both with Schema PCollection<Row> and without schema like PCollection<TInput>. That way it will be general purpose.
> Transforms for Intersect, Difference and Commons
> -------------------------------------------------
>
> Key: BEAM-9825
> URL: https://issues.apache.org/jira/browse/BEAM-9825
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Darshan Jani
> Assignee: Darshan Jani
> Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> I'd like to propose following new high-level transforms.
> * Intersect
> Compute the intersection between elements of two PCollection.
> Given _leftCollection_ and _rightCollection_, this transform returns a collection containing elements that common to both _leftCollection_ and _rightCollection_
>
> * Difference
> Compute the difference between elements of two PCollection.
> Given _leftCollection_ and _rightCollection_, this transform returns a collection containing elements that are in _leftCollection_ but not in _rightCollection_
> * Commons
> Find the elements that are commons to two PCollection, similar like the Unix
> comm utility.
> Given _leftCollection_ and rightCollection, this transform returns a CommonsResults with following:
> # elements only in _leftCollection_
> # elements only in _rightCollection_
> # elements in both collections
> I would like to work on this changes and submit a PR.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)