You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by HeartSaVioR <gi...@git.apache.org> on 2016/10/24 12:44:24 UTC
[GitHub] storm pull request #1743: STORM-2148 [Storm SQL] Trident mode: back to code ...
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1743
STORM-2148 [Storm SQL] Trident mode: back to code generate and compile Trident topology
This is on top of STORM-1446 (#1736).
Instead of fully generating topology code, I just generate classes for each expression, and let EvaluateXXX calls instance's method. This is much simpler and nearly no overhead added, and also much faster than evaluating code block.
I did a [simple benchmark](https://issues.apache.org/jira/browse/STORM-2148?focusedCommentId=15601152&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15601152) between evaluating code block and calling instance's method. Please refer the link if you are interested.
Also manually tested.
You can narrow the change to only last commit. It's 700 lines of change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-2148-on-top-of-STORM-1446
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1743.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1743
----
commit e1da468267d73c50ddc8c11ff8b3444949543c63
Author: Jungtaek Lim <ka...@gmail.com>
Date: 2016-10-13T10:00:10Z
STORM-1446 Compile the Calcite logical plan to Storm Trident logical plan
* Port SamzaSQL implementation to Storm
* https://github.com/milinda/samza-sql
* Apply some rules to optimize
* optimize Calc
* merge filter and projection scripts into one
* also applying short circuit
* Modify Trident unit tests to use new query planner
* arrange some files
* Move some files which are only used from standalone
* Remove some files which are no longer used
* guard the possibility of stack overflow error on explaining
* just leave error logs, and print out empty plan and continue
* reported this behavior to Calcite community
* leave some comments to clarify what it means
commit 68b80fae752f5f7d2824f50dec195d03f3520d6f
Author: Jungtaek Lim <ka...@gmail.com>
Date: 2016-10-24T12:27:07Z
STORM-2148 [Storm SQL] Trident mode: back to code generate and compile Trident topology
* Change RexNodeToBlockStatementCompiler (also rename)
* For standalone mode, it still returns BlockStatement which contains code block
* For Trident mode, it returns String which is a "class" implementation
* Replace evaluation of code block to instance method call
* Package compiled classes to be included to jar
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1743: STORM-2148 [Storm SQL] Trident mode: back to code generat...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on the issue:
https://github.com/apache/storm/pull/1743
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm issue #1743: STORM-2148 [Storm SQL] Trident mode: back to code generat...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/1743
Rebased onto master since STORM-1446 is merged into master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request #1743: STORM-2148 [Storm SQL] Trident mode: back to code ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/1743
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---