You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@samza.apache.org by Prateek Maheshwari <pr...@utexas.edu> on 2018/08/06 23:20:20 UTC

[DISCUSS] SEP-14: System and Stream Descriptors

Hi all,

Here's the proposal for System and Stream Descriptors - a way of
specifying systems, input and output streams properties in application
code instead of configurations.
https://cwiki.apache.org/confluence/display/SAMZA/SEP-14%3A+System+and+Stream+Descriptors

Here's the PR with an implementation for the proposal above:
https://github.com/apache/samza/pull/603

Please let me know if you have any questions or feedback for the SEP above.

Thanks,
Prateek

Re: [DISCUSS] SEP-14: System and Stream Descriptors

Posted by Prateek Maheshwari <pm...@linkedin.com>.

Hi folks,

I updated SEP-14 based on some feedback and discussions on the PR. Major changes are:

1. We will not support system level serdes _out of the box_ (e.g., for GenericSystemDescritors). Users must specify a stream level serde when creating Input/OutputStreamDescriptors. System implementers may still choose to support system level serdes for their own descriptors.

2. We will not support stream level input transformers in the initial implementation. We can add them later if we find this to be a common use case. 

3. We changed the SystemDescriptor base classes to optional interfaces for providing input and output descriptors. This allows system implementers more flexibility in providing input and output descriptors to users.

Please let me know if you have any questions or suggestions. If not, we can move SEP-14 from discussion to voting.

Thanks,
Prateek

> On Aug 6, 2018, at 4:20 PM, Prateek Maheshwari <pr...@utexas.edu> wrote:
> 
> Hi all,
> 
> Here's the proposal for System and Stream Descriptors - a way of
> specifying systems, input and output streams properties in application
> code instead of configurations.
> https://cwiki.apache.org/confluence/display/SAMZA/SEP-14%3A+System+and+Stream+Descriptors
> 
> Here's the PR with an implementation for the proposal above:
> https://github.com/apache/samza/pull/603
> 
> Please let me know if you have any questions or feedback for the SEP above.
> 
> Thanks,
> Prateek