You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrql.apache.org by "Leonidas Fegaras (JIRA)" <ji...@apache.org> on 2016/07/06 20:47:10 UTC
[jira] [Created] (MRQL-90) Add support for Storm streaming
Leonidas Fegaras created MRQL-90:
------------------------------------
Summary: Add support for Storm streaming
Key: MRQL-90
URL: https://issues.apache.org/jira/browse/MRQL-90
Project: MRQL
Issue Type: New Feature
Components: Streaming
Affects Versions: 0.9.8
Reporter: Leonidas Fegaras
Assignee: Leonidas Fegaras
This task is assigned to Achyut Paudel achyut.paudel2@mavs.uta.edu as part of his MS thesis at UTA. This extension will allow MRQL queries to run on Apache Storm DSPE (distributed stream processing engine). The stream unit is a Tuple with just one component: a Bag<MRData>, which is a buffer of MRData used for batch streaming. Each MRQL physical operator will correspond to 1-3 Storm bolts, while a source operator will correspond to a spout. For example, the MapReduce(m,r,X) operation will consist of 2 bolts: a bolt with a map function m (a Bolt subclass with an attribute m) that applies m to the input data, and another similar bolt with the reduce function r. These two bolts will be connected in the Storm topology with a Storm group-by operation. The query evaluator will first generate a Storm topology from a physical plan and then will use the Storm evaluator to evaluate the topology. The only new components needed for these extensions are: 1) Bolt and Spout subclasses to capture MRQL operators 2) a Storm topology generator from MRQL plans.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)