You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Milinda Lakmal Pathirage (JIRA)" <ji...@apache.org> on 2014/12/02 20:25:12 UTC
[jira] [Commented] (SAMZA-483) A common representation of
relational algebra for streaming SQL
[ https://issues.apache.org/jira/browse/SAMZA-483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231959#comment-14231959 ]
Milinda Lakmal Pathirage commented on SAMZA-483:
------------------------------------------------
I found following papers on universal calculus for stream processing languages which may be interesting to us when designing intermediate layer which every DSL/framework transformed/compiled into.
[1] A Universal Calculus for Stream Processing Languages - http://www.cs.bilkent.edu.tr/~bgedik/homepage/lib/exe/fetch.php/wiki:pubs:brooklet-calculus.pdf
[2] From a Calculus to an Execution Environment for Stream Processing - http://ftp.cs.nyu.edu/rgrimm/papers/debs12.pdf
[3] Distributed CQL Made Easy - http://www.cs.nyu.edu/rgrimm/papers/nedb11.pdf (summary), https://www.cs.nyu.edu/~soule/nedb11-talk.pdf (talk)
> A common representation of relational algebra for streaming SQL
> ----------------------------------------------------------------
>
> Key: SAMZA-483
> URL: https://issues.apache.org/jira/browse/SAMZA-483
> Project: Samza
> Issue Type: Sub-task
> Reporter: Yi Pan (Data Infrastructure)
> Priority: Minor
> Labels: project
>
> Per discussion with [~criccomini] and [~milinda], we agreed that it seems to be a good idea to define a common representation of relational algebra on top of the operators defined in the operator layer (see SAMZA-482), which can be the common base that we can use to generate the description/configuration of a Samza job.
> This common layer can also be used by DSL-like language parser as a result of parsing a DSL program.
> Some additional requirements needed in addition to pure relational algebra:
> 1) the common representation should include window operators and stream operators (i.e. IStream/DStream/RStream)
> 2) the common representation should include description on parallelism of the jobs (i.e. how many partitions the resultant Samza job will use)
> Some references:
> http://web.cs.wpi.edu/~mukherab/i/DCAPE.pdf
> https://cs.uwaterloo.ca/~david/cs848/stream-cql.pdf
> http://davis.wpi.edu/dsrg/PROJECTS/CAPE/publications.htm
> http://davis.wpi.edu/dsrg/PROJECTS/CAPE/slides.htm
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)