You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrql.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/07/15 22:17:20 UTC
[jira] [Commented] (MRQL-92) Use outer-joins for incremental
queries in Spark streaming mode
[ https://issues.apache.org/jira/browse/MRQL-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380218#comment-15380218 ]
ASF GitHub Bot commented on MRQL-92:
------------------------------------
GitHub user fegaras opened a pull request:
https://github.com/apache/incubator-mrql/pull/22
[MRQL-92] Use outer-joins for incremental queries in Spark streaming …
…mode
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fegaras/incubator-mrql MRQL-92
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-mrql/pull/22.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22
----
commit baf8301cc8f98bcf745cda5a0ef875b98ab00204
Author: Leonidas Fegaras <fe...@cse.uta.edu>
Date: 2016-07-15T22:14:51Z
[MRQL-92] Use outer-joins for incremental queries in Spark streaming mode
----
> Use outer-joins for incremental queries in Spark streaming mode
> ---------------------------------------------------------------
>
> Key: MRQL-92
> URL: https://issues.apache.org/jira/browse/MRQL-92
> Project: MRQL
> Issue Type: Improvement
> Components: Run-Time/Spark, Streaming
> Affects Versions: 0.9.8
> Reporter: Leonidas Fegaras
> Assignee: Leonidas Fegaras
> Priority: Critical
>
> Currently, incremental queries use Spark's coGroup to merge the current state with the results of processing the new data in the stream. With this patch, the merge is done with a special outer join that doesn't shuffle the state again (it only shuffles the results from the new data).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)