You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Parth-Brahmbhatt <gi...@git.apache.org> on 2014/12/12 19:45:26 UTC
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
GitHub user Parth-Brahmbhatt opened a pull request:
https://github.com/apache/storm/pull/344
Storm-444: Moving AutoHDFS to storm-hdfs and AutoHBase to storm-hbase. Added code to leverage delegation tokens in storm-hdfs and storm-hbase bolts/states.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Parth-Brahmbhatt/incubator-storm STORM-444
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/344.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #344
----
commit 65e9f0c814b2cddc772880042259b66194fd6fb7
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-05T22:48:34Z
STORM-586: TridentKafkaEmitter should catch updateOffsetException.
commit 5a67957efd3c874cd552e991d27eb9a8021129b3
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-03T22:41:24Z
Moving AutoHDFS to storm-hdfs. Added code to leverage delegation tokens in storm-hdfs bolts/states.
commit fefaf9bb1cab8890c19b538d8ea74c80ab30a853
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-11T01:50:32Z
making storm-hdfs bolt/states work with delegation tokens.
commit a37def89645f62a5df88ac6b46de3743703bb051
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-11T01:51:48Z
Moving AutoHDFS andAutoHBase to storm-hdfs and storm-hbase.Making storm-hbase bnd storm-hdfsolts/states work with delegation tokens.
commit 0de45c2e5badf211d5e9149801862c8cae8ec36c
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-11T04:44:58Z
adding documentation for AutoHbase and AutoHDFS usage.
commit b7b20fc84836d5a4b11ba343453a7607900cf595
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-11T04:46:04Z
Removing unused configs.
commit e590a7272bbb4838cbc2a9acd524e34094af3d27
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-12T04:13:20Z
moving the token adding to populate subject so on renewal the tokens are automatically added to UGI. Logging into hadoop again before renewal attempt to ensure other hadoop log-ins do not interfere.
commit 88a8658c619b42ac63b3371199f57969aeb5edb0
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-12T04:25:35Z
Removing unused imports, deleting accidentaly added file.
commit 5c696778c1c70e6cece4e09ee6434e2b6f96e16f
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-12T18:41:22Z
STORM-444 removing unused imports and unintended changes.
commit b272f35bed7688427b2bb88904f632ff7cd01d82
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2014-12-12T18:43:06Z
Revert " STORM-586: TridentKafkaEmitter should catch updateOffsetException."
This reverts commit 65e9f0c814b2cddc772880042259b66194fd6fb7.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/344
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/344#issuecomment-66816680
Tested with both storm-hdfs and storm-hbase bolts in a secure cluster and was able to leverage tokens. Also tested storm-hbase token expiration and auto renewal by setting "hbase.auth.token.max.lifetime" to 30 seconds. I could not find equivalent config for hdfs but I was able to test that hdfs tokens were being renewed at "nimbus.credential.renewers.freq.secs" and were pushed to topology.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on the pull request:
https://github.com/apache/storm/pull/344#issuecomment-67888850
@revans2 When you have time , can you please review this. as per @nathanmarz 's suggestion on the previous PR I moved AutoHDFS and AutoHBase to storm-hdfs and storm-hbase and also modified the bolts/trident states so they can work with delegation tokens when available.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the pull request:
https://github.com/apache/storm/pull/344#issuecomment-67895275
It looks fine to me +1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Storm-444: Moving AutoHDFS to storm-hdfs and A...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/344#issuecomment-67431236
+1. tested it on secure env. looks good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---