You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by NareshKosgi <gi...@git.apache.org> on 2014/10/27 17:34:42 UTC
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
GitHub user NareshKosgi opened a pull request:
https://github.com/apache/storm/pull/303
STORM-456:Storm UI cannot navigate to topology page when name contains spaces
1. The issue is due to space being replaced by "+" by the framework
2. In the routes function for core.clj I added a check for search and replace for "+" for the id being passed
3. I tested by changing the word count example and is working as I can click on the "word count" topology and get to the next page but the next page errors out with the following exception:
NotAliveException(msg:word+count-1-1414427540)
at backtype.storm.generated.Nimbus$getTopologyInfo_result.read(Nimbus.java:11347)
at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)
at backtype.storm.generated.Nimbus$Client.recv_getTopologyInfo(Nimbus.java:491)
at backtype.storm.generated.Nimbus$Client.getTopologyInfo(Nimbus.java:478)
at backtype.storm.ui.core$topology_page.invoke(core.clj:628)
at backtype.storm.ui.core$fn__8177.invoke(core.clj:853)
at compojure.core$make_route$fn__6356.invoke(core.clj:93)
at compojure.core$if_route$fn__6344.invoke(core.clj:39)
at compojure.core$if_method$fn__6337.invoke(core.clj:24)
at compojure.core$routing$fn__6362.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6366.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6391.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8216.invoke(core.clj:909)
at ring.middleware.keyword_params$wrap_keyword_params$fn__7033.invoke(keyword_params.clj:27)
at ring.middleware.nested_params$wrap_nested_params$fn__7072.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__7005.invoke(params.clj:55)
at ring.middleware.multipart_params$wrap_multipart_params$fn__7100.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7281.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7270.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7201.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7481.invoke(jetty.clj:16)
at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
I'm guessing I need to do a search and replace for the m as well and not just id? I know this pull request is not complete, I wanted to create a pull request and make sure I'm taking the correct approach to resolving this ticket.
P.S
I changed word count topology just for testing purpose, I will revert it back once I complete the testing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NareshKosgi/incubator-storm storm-456
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/303.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #303
----
commit e72a94c1de7dd4fa7744c9dd914439bcd59a634b
Author: Naresh Kosgi <na...@gmail.com>
Date: 2014-10-18T02:57:52Z
routes look for the plus sign
commit ccbfad1fec3f99531a8160dae51419abc3e5bb87
Author: Naresh Kosgi <na...@gmail.com>
Date: 2014-10-27T16:12:18Z
using word count to test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by NareshKosgi <gi...@git.apache.org>.
Github user NareshKosgi closed the pull request at:
https://github.com/apache/storm/pull/303
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-61668317
@NareshKosgi I am -1 on current PR. Please look at the @Parth-Brahmbhatt suggested approach.
Also lets not modify the existing examples to test it. It will be great if you can add unit tests to check if the different topology names are valid or not. Thanks.
@clockfly special characters seems to have no issue. Please check the attached image. I was able to submit ./bin/storm jar examples/storm-starter/storm-starter-topologies-0.9.3-rc2-SNAPSHOT.jar storm.starter.WordCountTopology "word#&count)("
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-60635672
search and replace is not a good solution, this seems to be another encoding decoding issue. I tested locally and from the initial investigation it feels like we have out of sync encoding and decoding schemes.
word count -> encoded by -> https://github.com/apache/storm/blob/master/storm-core/src/clj/backtype/storm/util.clj#L808 -> results in word+count.
however the ring gives us decoded id at https://github.com/apache/storm/blob/master/storm-core/src/clj/backtype/storm/ui/core.clj#L852 which does not transalte "word+count" to "word count".
We need to ensure our encoding and decoding is done with the same scheme.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by NareshKosgi <gi...@git.apache.org>.
GitHub user NareshKosgi reopened a pull request:
https://github.com/apache/storm/pull/303
STORM-456:Storm UI cannot navigate to topology page when name contains spaces
1. The issue is due to space being replaced by "+" by the framework
2. In the routes function for core.clj I added a check for search and replace for "+" for the id being passed
3. I tested by changing the word count example and is working as I can click on the "word count" topology and get to the next page but the next page errors out with the following exception:
NotAliveException(msg:word+count-1-1414427540)
at backtype.storm.generated.Nimbus$getTopologyInfo_result.read(Nimbus.java:11347)
at org.apache.thrift7.TServiceClient.receiveBase(TServiceClient.java:78)
at backtype.storm.generated.Nimbus$Client.recv_getTopologyInfo(Nimbus.java:491)
at backtype.storm.generated.Nimbus$Client.getTopologyInfo(Nimbus.java:478)
at backtype.storm.ui.core$topology_page.invoke(core.clj:628)
at backtype.storm.ui.core$fn__8177.invoke(core.clj:853)
at compojure.core$make_route$fn__6356.invoke(core.clj:93)
at compojure.core$if_route$fn__6344.invoke(core.clj:39)
at compojure.core$if_method$fn__6337.invoke(core.clj:24)
at compojure.core$routing$fn__6362.invoke(core.clj:106)
at clojure.core$some.invoke(core.clj:2443)
at compojure.core$routing.doInvoke(core.clj:106)
at clojure.lang.RestFn.applyTo(RestFn.java:139)
at clojure.core$apply.invoke(core.clj:619)
at compojure.core$routes$fn__6366.invoke(core.clj:111)
at ring.middleware.reload$wrap_reload$fn__6391.invoke(reload.clj:14)
at backtype.storm.ui.core$catch_errors$fn__8216.invoke(core.clj:909)
at ring.middleware.keyword_params$wrap_keyword_params$fn__7033.invoke(keyword_params.clj:27)
at ring.middleware.nested_params$wrap_nested_params$fn__7072.invoke(nested_params.clj:65)
at ring.middleware.params$wrap_params$fn__7005.invoke(params.clj:55)
at ring.middleware.multipart_params$wrap_multipart_params$fn__7100.invoke(multipart_params.clj:103)
at ring.middleware.flash$wrap_flash$fn__7281.invoke(flash.clj:14)
at ring.middleware.session$wrap_session$fn__7270.invoke(session.clj:43)
at ring.middleware.cookies$wrap_cookies$fn__7201.invoke(cookies.clj:160)
at ring.adapter.jetty$proxy_handler$fn__7481.invoke(jetty.clj:16)
at ring.adapter.jetty.proxy$org.mortbay.jetty.handler.AbstractHandler$0.handle(Unknown Source)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
I'm guessing I need to do a search and replace for the m as well and not just id? I know this pull request is not complete, I wanted to create a pull request and make sure I'm taking the correct approach to resolving this ticket.
P.S
I changed word count topology just for testing purpose, I will revert it back once I complete the testing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NareshKosgi/incubator-storm storm-456
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/303.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #303
----
commit fe5f41aa8332700d3f98422cb7d986fc47289bcd
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-05-21T16:03:11Z
STORM-216: Added Authentication and Authorization.
This is an upmerged version of https://github.com/yahoo/incubator-storm/tree/security
commit ce6e5d26384d7f5d831b35e4eff126fe214981d1
Author: Derek Dagit <de...@yahoo-inc.com>
Date: 2014-05-22T18:34:23Z
rename test for consistent capitalization
commit 698bb9c9788b82d4127d861fb3ecf06a06b683c2
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-05-28T13:45:22Z
Merge branch 'master' into security
commit 6592b8209c49a98db15b3d6d228f488aa6c2e623
Author: Kishor Patil <pa...@yahoo.com>
Date: 2014-06-09T15:42:39Z
Add missing ACLs to error znodes and remove auto vivification of error znodes
commit cf2e8b7ee06b455a90bd4b3bfd53facef1369612
Author: Sriharsha Chintalapani <ma...@harsha.io>
Date: 2014-06-10T22:01:33Z
Storm 344. (Security) nimbus renew-credentials not calling ICredentialsRenewer.renew
commit 0a98bee214b46ed20b566a9b49c3eca2895f9fd5
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-11T16:07:59Z
Merge branch 'master' into security-upmerge
Conflicts:
.gitignore
storm-core/src/clj/backtype/storm/daemon/drpc.clj
storm-core/src/clj/backtype/storm/daemon/executor.clj
storm-core/src/clj/backtype/storm/daemon/logviewer.clj
storm-core/src/clj/backtype/storm/daemon/worker.clj
storm-core/src/clj/backtype/storm/timer.clj
storm-core/src/clj/backtype/storm/ui/core.clj
storm-core/src/clj/backtype/storm/ui/helpers.clj
storm-core/src/clj/backtype/storm/util.clj
storm-core/src/jvm/backtype/storm/Config.java
storm-core/src/jvm/backtype/storm/utils/Utils.java
commit 118b9221b492ed8b91e6633c3cfb748bc1b82790
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-11T16:10:04Z
Merge branch 'master' into security
commit 2131a0aeb9074b2c83a09d7515ff8e8ae86f6eaf
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-12T15:58:13Z
Added back in the user to the web ui.
commit 41615b3c4b174077ac1c729af4aef32e5b79d3c5
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-12T15:59:46Z
Merge branch 'master' into security
commit bc91ed88d77e392f38c406d143e7ac37bc634564
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-12T16:01:35Z
Added license to UI template.
commit a762f1c5f99a7a9e77038399f0f14ae03b1414c7
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-12T17:48:51Z
Merge branch 'STORM-344' of https://github.com/harshach/incubator-storm into STORM-344
STORM-344: (Security) nimbus renew-credentials not calling ICredentialsRenewer.renew
commit 92e3a5742374a3a7c3aae20cbeda32ce7b033526
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-12T21:09:57Z
Merge branch 'master' into security-upmerge
Conflicts:
storm-core/src/clj/backtype/storm/LocalCluster.clj
storm-core/src/clj/backtype/storm/cluster.clj
storm-core/src/clj/backtype/storm/config.clj
storm-core/src/clj/backtype/storm/daemon/drpc.clj
storm-core/src/clj/backtype/storm/testing.clj
storm-core/src/clj/backtype/storm/testing4j.clj
storm-core/src/clj/backtype/storm/thrift.clj
storm-core/src/clj/backtype/storm/ui/core.clj
storm-core/src/clj/backtype/storm/util.clj
storm-core/src/clj/backtype/storm/zookeeper.clj
commit ab7784e49d251ca4da967c6ec6bc340cc7f940aa
Author: Kishor Patil <pa...@yahoo.com>
Date: 2014-06-17T15:19:00Z
Force free a slot in bad-state
commit d1ba4fc4acdadd5e5e138395bdc5892dfdb88bff
Author: Derek Dagit <de...@yahoo-inc.com>
Date: 2014-06-17T15:56:51Z
Do not clean up user file when rmr is unsuccessful
commit 87cdbf5fdf5bfb49b983604542283f05123d0d51
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-17T18:32:18Z
Merge branch 'STORM-357' of https://github.com/d2r/incubator-storm into STORM-357
STORM-357: Cleans workers-users file only when rmr is successful
commit ea946d04dcb6df8e65dbf16500a361eaaba13432
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-18T23:58:33Z
Show node details on errors for STORM-360 on security
commit 79089ad0da80e38eb36b7ea91be8b43795dc4efb
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-19T21:04:02Z
Merge tag 'v0.9.2-incubating' into security
[maven-release-plugin] copy for tag v0.9.2-incubating
Conflicts:
storm-core/pom.xml
commit f20df7d52d52abc9bc03a0cf45388241927cff5a
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-20T20:35:16Z
Fixing coding style and component template
commit c545b9d638067c0ae4528e16f14e67c56e0dd47e
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-20T23:01:00Z
Fix nimbus use of doto
commit d7c1d1d0a909079a370ed35aaac91668eef33a22
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-23T14:31:41Z
Merge branch 'node-on-error-security' of https://github.com/kishorvpatil/incubator-storm into STORM-360-security
STORM-360: Add node details for Error Topology and Component pages on security
commit 65aee65af54dd29434af8f5ee403233b597561b6
Author: Robert (Bobby) Evans <bo...@apache.org>
Date: 2014-06-23T15:18:38Z
Merge branch 'master' into security
Conflicts:
storm-core/src/clj/backtype/storm/cluster.clj
storm-core/src/clj/backtype/storm/ui/core.clj
storm-core/test/clj/backtype/storm/cluster_test.clj
commit 357fd09dff8eec1f224d7f00f5344931ec2b64df
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-23T16:13:34Z
Add color span to most recent error and fix ui templates for security.
commit 28c168fd7d0272f88d586f6f572eab937b874f22
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-24T19:12:56Z
Add check for empty table before sorting on security
commit 8325186b731259c17d4b06aa2c288ffc4f6dc59b
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-25T15:32:17Z
revert tablesorter fix
commit 7bb8ec6d8068fe2eeec65085bfa0140aca0fb0a9
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-06-26T00:13:03Z
Use javascript to color-code recent errors
commit f3112fa76e98038da83170513a476499dde0bb41
Author: Sriharsha Chintalapani <ma...@harsha.io>
Date: 2014-06-26T22:05:21Z
STORM-347. (Security) authentication should allow for groups not just users.
commit 3c6930dfe4447b6077916b9f9a07b062141b5305
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-07-07T18:36:49Z
AutoHDFS for getting HDFS delegation token and auto renew.
commit 00e80e9a132764d4b73737d2f7a52282e5247856
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-07-07T18:43:59Z
Merge remote-tracking branch 'upstream/security' into security
commit e04c37356c96d9851c00542c739d053e4bf36481
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-07-07T18:49:42Z
Revert "AutoHDFS for getting HDFS delegation token and auto renew."
This reverts commit 3c6930dfe4447b6077916b9f9a07b062141b5305.
commit ae5d0c312d170c883e9758767808e1c8d33f58c8
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2014-07-08T22:08:33Z
Use TBinaryProtocol without max buffer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-62628045
Have you tested with some other topologies, We were doing decoding before but it was removed because ring already decodes it before calling this function. Try a topology with name test%5Ftest .
See https://issues.apache.org/jira/browse/STORM-439
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by NareshKosgi <gi...@git.apache.org>.
Github user NareshKosgi closed the pull request at:
https://github.com/apache/storm/pull/303
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by NareshKosgi <gi...@git.apache.org>.
Github user NareshKosgi commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-62625740
Turns out this was just a simple change. Just needed to add url decode function. Updated the PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-61631456
@NareshKosgi,
Have you considered other special character that may used as topology name?
Like the table, #, &, (, and etc..
Is it possble that this fix only works for "space", but fails for "table"?
Another approach is to add strict name checking when submitting topology. This may make life easier?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...
Posted by NareshKosgi <gi...@git.apache.org>.
Github user NareshKosgi commented on the pull request:
https://github.com/apache/storm/pull/303#issuecomment-61671428
@harshach Working on exactly those changes. Will have a updated PR in a few days.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---