[jira] [Updated] (QUARKS-91) Support joining a stream against another stream/window

["Will Marshall (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/QUARKS-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Will Marshall updated QUARKS-91:
--------------------------------
    Description: 
Quarks should support the ability to join a stream against another stream or window. The interface would look something like this:
{code}
    <J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window, BiFunction<T, List<U>, J> joiner);
    
    <J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J> joiner);

    <J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream, BiFunction<T, U, J> joiner);
{code}

For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB, joiner)* would call *joiner* on every tuple on streamA with the contents of the partition associated with the key *keyer.apply(tuple)*.

For two streams, streamA and streamB: *streamA.joinLast(keyer, streamB, joiner)* would call the supplied *joiner* function on the most recent value of streamA and the most recent value of streamB whose key matched *keyer.apply(tuple)*. This happens whenever a tuple passes through streamA.

For two streams, streamA and streamB: *streamA.joinLast(streamB, joiner)* would call the supplied *joiner* function on the most recent values of streamA and streamB whenever a tuple passes through streamA.

*Issues:*
- The definition of Oplet<I, O> requires that all input tuples be of the same type. If streamA and streamB are of different types, it's not clear how to create a "join" oplet where one input port has the tuples of streamA and another has the tuples of streamB.
- The windowing library doesn't currently support accessing the contents of a partition from a quarks.window.Window reference.

  was:
Quarks should support the ability to join a stream against another stream or window. The interface would look something like this:
{code}
    <J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window, BiFunction<T, List<U>, J> joiner);
    
    <J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J> joiner);

    <J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream, BiFunction<T, U, J> joiner);
{code}

For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB, joiner)* would call *joiner* on every tuple on streamA with the contents of the partition associated with the key *keyer.apply(tuple)*.

For two streams streamA and streamB: *streamA.joinLast(keyer, streamB, joiner)* would call the supplied *joiner* function on the most recent value of streamA and the most recent value of streamB whose key matched *keyer.apply(tuple)*. This happens whenever a tuple passes through streamA.

For two streams streamA and streamB: *streamA.joinLast(streamB, joiner)* would call the supplied *joiner* function on the most recent values of streamA and streamB whenever a tuple passes through streamA.

*Issues:*
- The definition of Oplet<I, O> requires that all input tuples be of the same type. If streamA and streamB are of different types, it's not clear how to create a "join" oplet where one input port has the tuples of streamA and another has the tuples of streamB.
- The windowing library doesn't currently support accessing the contents of a partition from a quarks.window.Window reference.


> Support joining a stream against another stream/window
> ------------------------------------------------------
>
>                 Key: QUARKS-91
>                 URL: https://issues.apache.org/jira/browse/QUARKS-91
>             Project: Quarks
>          Issue Type: Bug
>            Reporter: Will Marshall
>            Assignee: Will Marshall
>
> Quarks should support the ability to join a stream against another stream or window. The interface would look something like this:
> {code}
>     <J, U, K> TStream<J> join(Function<T, K> keyer, TWindow<U, K> window, BiFunction<T, List<U>, J> joiner);
>     
>     <J, U, K> TStream<J> joinLast(TStream<U> lastStream, BiFunction<T, U, J> joiner);
>     <J, U, K> TStream<J> joinLast(Function<T, K> keyer, TStream<U> lastStream, BiFunction<T, U, J> joiner);
> {code}
> For a stream and a window, streamA and windowB: *streamA.join(keyer, windowB, joiner)* would call *joiner* on every tuple on streamA with the contents of the partition associated with the key *keyer.apply(tuple)*.
> For two streams, streamA and streamB: *streamA.joinLast(keyer, streamB, joiner)* would call the supplied *joiner* function on the most recent value of streamA and the most recent value of streamB whose key matched *keyer.apply(tuple)*. This happens whenever a tuple passes through streamA.
> For two streams, streamA and streamB: *streamA.joinLast(streamB, joiner)* would call the supplied *joiner* function on the most recent values of streamA and streamB whenever a tuple passes through streamA.
> *Issues:*
> - The definition of Oplet<I, O> requires that all input tuples be of the same type. If streamA and streamB are of different types, it's not clear how to create a "join" oplet where one input port has the tuples of streamA and another has the tuples of streamB.
> - The windowing library doesn't currently support accessing the contents of a partition from a quarks.window.Window reference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)