You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/04/18 21:52:41 UTC
[jira] [Commented] (SAMZA-1204) Visualize StreamGraph and
ExecutionPlan
[ https://issues.apache.org/jira/browse/SAMZA-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973584#comment-15973584 ]
ASF GitHub Bot commented on SAMZA-1204:
---------------------------------------
GitHub user xinyuiscool opened a pull request:
https://github.com/apache/samza/pull/127
SAMZA-1204: Visualize StreamGraph and ExecutionPlan
First look: https://xinyuiscool.github.io/visualizer/plan.html. This is based on the example graph JSON generated in TestJobGraphJsonGenerator.
For the api-test job:
For the join-test job:
How to visualize the plan (locally):
Once a Samza application (using fluent API) is deployed, an execution plan will be generated by the ExecutionPlanner. The plan JSON will be written to a file (plan.json) under the bin directory, which also contains the plan.html and javscripts (js folder). So the bin dir has the following for visualization:
./bin
- plan.json
- plan.html
- /js
- d3.js
- dagre.js
- planToDagre.js
Then deployed locally, the user can directly view the plan by enter the following in a browser:
**file:///my/samza/app/dir/bin/plan.html**
To visualize the plan remotely, the user can either scp the files into local box and view the plan there, or bring up a simple http server to serve the page.
In the future when we have the Samza dashboard, we can add the plan visualization there to avoid the above manual steps.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xinyuiscool/samza SAMZA-1204
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/127.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #127
----
commit 0079c6d78e22de82c2857b20d8ae5e7436d38efa
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-10T23:53:19Z
SAMZA-1204: Visualize StreamGraph and ExecutionPlan
commit beaf1bde2dd2acbf891639ffd1d06a69e38ac19d
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-11T16:04:38Z
Minor adjustment to html
commit 40cf47d2cf08092acd4ddc540672ffce06768644
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-12T17:22:14Z
Add support to operator call trace
commit d78be41bd243c6fc5cafd95aaea289eac34b511f
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-15T01:12:32Z
Generate plan jon in the application runner
commit 16e3a4c6ee9d440418a6f40faefc1bbcafe9b893
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T19:03:28Z
Further enhancement of the visualizer
commit 6eca9f85947e25bd40617eb97a0bd87ae42d77fe
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T19:12:28Z
Merge branch 'master' into SAMZA-1204
commit 95d89211704e264533de6e0e9463f8f6d1bdc417
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T19:46:40Z
Add application name and id
commit d3c2d4249b3948bfaf12828cbdb9b3a47b609157
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T21:47:54Z
Update page title
commit 5e40fe5d3e17a5138b6ae7bcd64eeb28b1717fb5
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T23:00:11Z
Fetch partition info for json
commit 7166d690856451d5161d430c5347726953aa4f2a
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T23:26:36Z
Remove sample json
commit 7f447391e2b6a0eca04ac3990ad14dc015250a4f
Author: xinyuiscool <xi...@gmail.com>
Date: 2017-04-17T23:32:25Z
Use minified dagre
commit de3d43962306f5a213a83884c6d604cde821ada8
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-18T17:24:02Z
Clean up the scripts
commit 1459ba93a509441de3d8edde32ec4e16b9be3df5
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-18T17:24:57Z
Merge branch 'SAMZA-1204' of https://github.com/xinyuiscool/samza into SAMZA-1204
commit 4039ba2f9f36faa004b930887a804f7907b63594
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-18T19:14:44Z
Clean up based on gradle build
commit 54325ab7fa9484ad442b68a7e32a59358f2beb83
Author: Xinyu Liu <xi...@xiliu-ld.linkedin.biz>
Date: 2017-04-18T19:22:37Z
Small javadoc change
----
> Visualize StreamGraph and ExecutionPlan
> ---------------------------------------
>
> Key: SAMZA-1204
> URL: https://issues.apache.org/jira/browse/SAMZA-1204
> Project: Samza
> Issue Type: Bug
> Affects Versions: 0.13.0
> Reporter: Xinyu Liu
> Assignee: Xinyu Liu
> Fix For: 0.13.0
>
>
> Based on SAMZA-1178, we are able to generate the JSON representation of StreamGraph and ExecutionPlan. We need to provide a visualizer to render the json as DAG and let users to view it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)