You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com> on 2016/05/05 21:46:20 UTC
Review Request 47042: SAMZA-915: libear pipeline programming APIs -
part 1
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47042/
-----------------------------------------------------------
Review request for samza, Boris Shkolnik, Chris Pettitt, Chinmay Soman, Jake Maes, Navina Ramesh, Jagadish Venkatraman, and Xinyu Liu.
Bugs: SAMZA-915
https://issues.apache.org/jira/browse/SAMZA-915
Repository: samza
Description
-------
SAMZA-915: linear pipeline programming APIs
This is just laying out the skeleton for the linear pipeline programming APIs. Implementation classes are to be added in part-2
High-level description:
- Separate API classes that are exposed and used by user vs implementation classes in api and impl packages.
- Added initial draft of all StreamOperator classes that allows user to build the description of operators in fluent style
- Added basic skeleton implementation classes to actually create StreamPipeline
- Modified DataStreamTask and examples in test to illustrate the programming model
TODO:
- unit tests
Diffs
-----
samza-operator/src/main/java/org/apache/samza/operators/api/Data.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Filter.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/FlatMapper.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Mapper.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Operator.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Scanner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Schema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Sink.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Transformer.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Tuple.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/data/DataStream.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/join/Joiner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/join/PartialJoinOperator.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/window/TriggerBuilder.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/IncomingMessageTuple.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/avro/AvroData.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/avro/AvroSchema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlAvroSerde.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlAvroSerdeFactory.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlStringSerde.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlStringSerdeFactory.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/string/StringData.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/string/StringSchema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/exception/OperatorException.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/factory/DataStream.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/factory/StreamPipeline.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/ProcessorContext.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/SimpleOperatorImpl.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/StreamPipeline.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/join/Joiner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/scan/Scanner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Batch.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/SessionWindow.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Timeout.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Window.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/task/DataStreamTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/operators/data/serializers/SqlAvroSerdeTest.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamJoinTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamSplitTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamUserTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/47042/diff/
Testing
-------
./gradlew clean build passes locally
Thanks,
Yi Pan (Data Infrastructure)
Re: Review Request 47042: SAMZA-915: linear pipeline programming APIs
- part 1
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47042/
-----------------------------------------------------------
(Updated May 5, 2016, 9:49 p.m.)
Review request for samza, Boris Shkolnik, Chris Pettitt, Chinmay Soman, Jake Maes, Navina Ramesh, Jagadish Venkatraman, and Xinyu Liu.
Summary (updated)
-----------------
SAMZA-915: linear pipeline programming APIs - part 1
Bugs: SAMZA-915
https://issues.apache.org/jira/browse/SAMZA-915
Repository: samza
Description
-------
SAMZA-915: linear pipeline programming APIs
This is just laying out the skeleton for the linear pipeline programming APIs. Implementation classes are to be added in part-2
High-level description:
- Separate API classes that are exposed and used by user vs implementation classes in api and impl packages.
- Added initial draft of all StreamOperator classes that allows user to build the description of operators in fluent style
- Added basic skeleton implementation classes to actually create StreamPipeline
- Modified DataStreamTask and examples in test to illustrate the programming model
TODO:
- unit tests
Diffs
-----
samza-operator/src/main/java/org/apache/samza/operators/api/Data.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Filter.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/FlatMapper.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Mapper.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Operator.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Scanner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Schema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Sink.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Transformer.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/Tuple.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/data/DataStream.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/join/Joiner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/join/PartialJoinOperator.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/api/window/TriggerBuilder.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/IncomingMessageTuple.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/avro/AvroData.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/avro/AvroSchema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlAvroSerde.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlAvroSerdeFactory.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlStringSerde.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/serializers/SqlStringSerdeFactory.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/string/StringData.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/data/string/StringSchema.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/exception/OperatorException.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/factory/DataStream.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/factory/StreamPipeline.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/ProcessorContext.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/SimpleOperatorImpl.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/impl/StreamPipeline.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/join/Joiner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/scan/Scanner.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Batch.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/SessionWindow.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Timeout.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/operators/window/Window.java PRE-CREATION
samza-operator/src/main/java/org/apache/samza/task/DataStreamTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/operators/data/serializers/SqlAvroSerdeTest.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamJoinTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamSplitTask.java PRE-CREATION
samza-operator/src/test/java/org/apache/samza/task/DataStreamUserTask.java PRE-CREATION
Diff: https://reviews.apache.org/r/47042/diff/
Testing
-------
./gradlew clean build passes locally
Thanks,
Yi Pan (Data Infrastructure)