You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com> on 2015/05/14 11:49:39 UTC
Review Request 34206: WIP: update operator API to allow callbacks and
allow
a single API to trigger OperatorRouter execution w/ user callbacks
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/
-----------------------------------------------------------
Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
Bugs: SAMZA-552
https://issues.apache.org/jira/browse/SAMZA-552
Repository: samza
Description
-------
This is one version of the Operator API change:
- Merge RelationOperator and TupleOperator to SimpleOperator
- Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
- Modified test case examples to illustrate usage
The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and beforeSend() functions.
Diffs
-----
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/sql/operators/window/MyBoundedTimeWindow.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/34206/diff/
Testing
-------
Thanks,
Yi Pan (Data Infrastructure)
Re: Review Request 34206: WIP: update operator API to allow callbacks
and
allow a single API to trigger OperatorRouter execution w/ user callbacks
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
> On May 15, 2015, 4:27 p.m., Milinda Pathirage wrote:
> > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java, line 36
> > <https://reviews.apache.org/r/34206/diff/2/?file=960924#file960924line36>
> >
> > How about supporting multi column primary keys? We can add it to the API even if we are not going to support it at the begining.
Sure. I will change the API to return a list instead of a single value.
- Yi
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/#review83937
-----------------------------------------------------------
On May 15, 2015, 2:16 a.m., Yi Pan (Data Infrastructure) wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34206/
> -----------------------------------------------------------
>
> (Updated May 15, 2015, 2:16 a.m.)
>
>
> Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
>
>
> Bugs: SAMZA-552
> https://issues.apache.org/jira/browse/SAMZA-552
>
>
> Repository: samza
>
>
> Description
> -------
>
> This is one version of the Operator API change:
> - Merge RelationOperator and TupleOperator to SimpleOperator
> - Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
> - Modified test case examples to illustrate usage
>
> The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and afterProcess() functions.
>
>
> Diffs
> -----
>
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34206/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Yi Pan (Data Infrastructure)
>
>
Re: Review Request 34206: WIP: update operator API to allow callbacks
and
allow a single API to trigger OperatorRouter execution w/ user callbacks
Posted by Milinda Pathirage <mi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/#review83937
-----------------------------------------------------------
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java
<https://reviews.apache.org/r/34206/#comment134999>
How about supporting multi column primary keys? We can add it to the API even if we are not going to support it at the begining.
- Milinda Pathirage
On May 15, 2015, 2:16 a.m., Yi Pan (Data Infrastructure) wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34206/
> -----------------------------------------------------------
>
> (Updated May 15, 2015, 2:16 a.m.)
>
>
> Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
>
>
> Bugs: SAMZA-552
> https://issues.apache.org/jira/browse/SAMZA-552
>
>
> Repository: samza
>
>
> Description
> -------
>
> This is one version of the Operator API change:
> - Merge RelationOperator and TupleOperator to SimpleOperator
> - Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
> - Modified test case examples to illustrate usage
>
> The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and afterProcess() functions.
>
>
> Diffs
> -----
>
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34206/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Yi Pan (Data Infrastructure)
>
>
Re: Review Request 34206: WIP: update operator API to allow callbacks
and
allow a single API to trigger OperatorRouter execution w/ user callbacks
Posted by Milinda Pathirage <mi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/#review83938
-----------------------------------------------------------
Ship it!
I am +1 for the overall API design. Lets push this to samza-sql branch if others are okay with the patch.
- Milinda Pathirage
On May 15, 2015, 2:16 a.m., Yi Pan (Data Infrastructure) wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34206/
> -----------------------------------------------------------
>
> (Updated May 15, 2015, 2:16 a.m.)
>
>
> Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
>
>
> Bugs: SAMZA-552
> https://issues.apache.org/jira/browse/SAMZA-552
>
>
> Repository: samza
>
>
> Description
> -------
>
> This is one version of the Operator API change:
> - Merge RelationOperator and TupleOperator to SimpleOperator
> - Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
> - Modified test case examples to illustrate usage
>
> The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and afterProcess() functions.
>
>
> Diffs
> -----
>
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
> samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
> samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34206/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Yi Pan (Data Infrastructure)
>
>
Re: Review Request 34206: WIP: update operator API to allow callbacks
and
allow a single API to trigger OperatorRouter execution w/ user callbacks
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/
-----------------------------------------------------------
(Updated May 15, 2015, 2:16 a.m.)
Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
Bugs: SAMZA-552
https://issues.apache.org/jira/browse/SAMZA-552
Repository: samza
Description (updated)
-------
This is one version of the Operator API change:
- Merge RelationOperator and TupleOperator to SimpleOperator
- Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
- Modified test case examples to illustrate usage
The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and afterProcess() functions.
Diffs
-----
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/34206/diff/
Testing
-------
Thanks,
Yi Pan (Data Infrastructure)
Re: Review Request 34206: WIP: update operator API to allow callbacks
and
allow a single API to trigger OperatorRouter execution w/ user callbacks
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34206/
-----------------------------------------------------------
(Updated May 15, 2015, 12:59 a.m.)
Review request for samza, Yan Fang, Chris Riccomini, Guozhang Wang, Milinda Pathirage, Navina Ramesh, and Naveen Somasundaram.
Changes
-------
Unified two RB. This RB now include the callback-based version of APIs
Bugs: SAMZA-552
https://issues.apache.org/jira/browse/SAMZA-552
Repository: samza
Description
-------
This is one version of the Operator API change:
- Merge RelationOperator and TupleOperator to SimpleOperator
- Add OperatorRouter to extends from Operator to allow process() and refresh() the whole connected set of SimpleOperators
- Modified test case examples to illustrate usage
The downside of not defining a separate OperatorCallback function is that if there are some commonly used user functions to preprocess the input and outgoing messages, the user will have to extend different SimpleOperator classes to override the beforeProcess() and beforeSend() functions.
Diffs (updated)
-----
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Relation.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Table.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/data/Tuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/Operator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorCallback.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/RelationOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/SqlOperatorFactory.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/TupleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/operators/spec/OperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/api/router/OperatorRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/data/IncomingMessageTuple.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/NoopOperatorCallback.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperator.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorFactoryImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorImpl.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleOperatorSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/factory/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/join/StreamStreamJoin.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionOp.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/partition/PartitionSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/Join.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/relation/JoinSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStream.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/stream/InsertStreamSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/BoundedTimeWindow.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/operators/window/WindowSpec.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/router/SimpleRouter.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/sql/window/storage/OrderedStoreKey.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/LongOffset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/system/sql/Offset.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/OperatorMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/RouterMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SimpleMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/SqlMessageCollector.java PRE-CREATION
samza-sql-core/src/main/java/org/apache/samza/task/sql/StoreMessageCollector.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/RandomWindowOperatorTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/StreamSqlTask.java PRE-CREATION
samza-sql-core/src/test/java/org/apache/samza/task/sql/UserCallbacksSqlTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/34206/diff/
Testing
-------
Thanks,
Yi Pan (Data Infrastructure)