You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@livy.apache.org by GitBox <gi...@apache.org> on 2020/01/31 08:36:01 UTC
[GitHub] [incubator-livy] RogPodge opened a new pull request #278: Livy
kerberos and basic Auth for IT
RogPodge opened a new pull request #278: Livy kerberos and basic Auth for IT
URL: https://github.com/apache/incubator-livy/pull/278
## What changes were proposed in this pull request?
This PR configures the integration tests to work on Livy Servers instances that require basic or kerberos authentication. To do this we had to replace the async httpclient with apaches normal httpclient, as there were issues found getting kerberos authentication to work with the async client.
This PR also contains a minor fix to the InteractiveIT test suite. One of the tests would occasionally fail as a statement submitted to the interactive session would not be finished processing when the result was checked. The correctness check was also changed to correctly check for code completition
## How was this patch tested?
This patch was tested on local deployments of livy server, both on a instance that required kerberos authentication and on one that required basic authentication.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.26%
Complexity ? 964
=========================================
Files ? 104
Lines ? 5952
Branches ? 900
=========================================
Hits ? 4063
Misses ? 1310
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.48% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...fe5bfac](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `57.34%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.05%
Complexity ? 957
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4047
Misses ? 1317
Partials ? 583
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `73.49% <20%> (ø)` | `20 <1> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...6b9fa41](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] FourSwordKirby commented on a change in pull
request #278: [LIVY-748] Add support for running Livy Integration tests
against secure external clusters
Posted by GitBox <gi...@apache.org>.
FourSwordKirby commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r376842433
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/Cluster.scala
##########
@@ -45,20 +47,46 @@ trait Cluster {
def jdbcEndpoint: Option[String]
def hdfsScratchDir(): Path
+ def authScheme: String
+ def user: String
+ def password: String
+ def sslCertPath: String
+
def doAsClusterUser[T](task: => T): T
- lazy val hadoopConf = {
+ def initKubeConf(): Configuration = {
val conf = new Configuration(false)
configDir().listFiles().foreach { f =>
if (f.getName().endsWith(".xml")) {
conf.addResource(new Path(f.toURI()))
}
}
+ UserGroupInformation.setConfiguration(conf);
+ UserGroupInformation.loginUserFromKeytab(s"${user}@AZDATA.LOCAL",
+ s"/tests/kerberos_setup/${user}.keytab");
Review comment:
no those values shouldn't be hardcoded I just removed them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge opened a new pull request #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge opened a new pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278
## What changes were proposed in this pull request?
https://issues.apache.org/jira/projects/LIVY/issues/LIVY-748?filter=reportedbyme
This PR configures the integration tests to work on Livy Servers instances that require basic or kerberos authentication. To do this we had to replace the async httpclient with apaches normal httpclient, as there were issues found getting kerberos authentication to work with the async client.
This PR also contains a minor fix to the InteractiveIT test suite. One of the tests would occasionally fail as a statement submitted to the interactive session would not be finished processing when the result was checked. The correctness check was also changed to correctly check for code completition
## How was this patch tested?
This patch was tested on local deployments of livy server, both on a instance that required kerberos authentication and on one that required basic authentication.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.13%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4052
Misses ? 1314
Partials ? 581
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.03% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...26b1706](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `58.79%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.06%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4048
Misses ? 1320
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `73.49% <20%> (ø)` | `20 <1> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...26b1706](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-591189348
@jerryshao another review request
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `57.34%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
========================================
Coverage ? 65.1%
Complexity ? 932
========================================
Files ? 104
Lines ? 5947
Branches ? 899
========================================
Hits ? 3872
Misses ? 1528
Partials ? 547
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `94.2% <100%> (ø)` | `18 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.17% <100%> (ø)` | `37 <0> (?)` | |
| [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `73.49% <20%> (ø)` | `20 <1> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...c25361e](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384236391
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/BaseIntegrationTestSuite.scala
##########
@@ -65,7 +79,19 @@ abstract class BaseIntegrationTestSuite extends FunSuite with Matchers with Befo
protected def uploadToHdfs(file: File): String = {
val hdfsPath = new Path(cluster.hdfsScratchDir(),
UUID.randomUUID().toString() + "-" + file.getName())
- cluster.fs.copyFromLocalFile(new Path(file.toURI()), hdfsPath)
+
+ if(authScheme == "kerberos") {
+ val proxy =
+ UserGroupInformation.createProxyUser(user, UserGroupInformation.getLoginUser());
+ proxy.doAs(new PrivilegedExceptionAction[Unit] {
+ def run() = {
+ cluster.fs.copyFromLocalFile(new Path(file.toURI()), hdfsPath);
+ }
+ });
Review comment:
nit: please remove ";" in Scala code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.16%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4054
Misses ? 1314
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.03% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...6529692](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384700152
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/conf/cluster.spec.template
##########
@@ -0,0 +1,46 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# type of cluster you are running integration tests against
Review comment:
Moved to the resources folder
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384238435
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/LivyRestClient.scala
##########
@@ -75,14 +80,19 @@ class LivyRestClient(val httpClient: AsyncHttpClient, val livyEndpoint: String)
}
def snapshot(): SessionSnapshot = {
- val r = httpClient.prepareGet(url).execute().get()
- assertStatusCode(r, HttpServletResponse.SC_OK)
+ val httpGet = new HttpGet(url)
+ val r = httpClient.execute(httpGet)
+ assertStatusCode(r.getStatusLine(), HttpServletResponse.SC_OK)
- mapper.readValue(r.getResponseBodyAsStream, classOf[SessionSnapshot])
+ val sessionSnapshot = mapper.readValue(r.getEntity().getContent, classOf[SessionSnapshot])
+ r.close()
Review comment:
I think we should always close the request no matter the test is succeed or not, we should wrap the test code with the method like `tryWithFinal(xxxx).`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583906967
@jerryshao pinging again
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-590014034
Small ping for a review @jerryshao
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-588137762
@jerryshao
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-588399825
I added some documentation on how to run external tests. Does the PR look good now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `57.34%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
========================================
Coverage ? 65.1%
Complexity ? 932
========================================
Files ? 104
Lines ? 5947
Branches ? 899
========================================
Hits ? 3872
Misses ? 1528
Partials ? 547
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `94.2% <100%> (ø)` | `18 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.17% <100%> (ø)` | `37 <0> (?)` | |
| [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `73.49% <20%> (ø)` | `20 <1> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...c25361e](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r379953472
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/BaseIntegrationTestSuite.scala
##########
@@ -96,7 +121,30 @@ abstract class BaseIntegrationTestSuite extends FunSuite with Matchers with Befo
// Please create an issue if this breaks test logging for cluster creation.
protected override def beforeAll() = {
cluster = Cluster.get()
- httpClient = new AsyncHttpClient()
+ httpClient = new DefaultHttpClient()
+
+ if (authScheme == "kerberos") {
+ val use_jaas_creds = new Credentials() {
Review comment:
Can we please change to use camel style.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `60%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.23%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4058
Misses ? 1308
Partials ? 581
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...ain/java/org/apache/livy/rsc/driver/RSCDriver.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvUlNDRHJpdmVyLmphdmE=) | `81.66% <100%> (ø)` | `45 <0> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...3eeced5](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-586783779
I'm OK with the current changes, but I'm not sure how do we run the external cluster test, can you please add some comments or docs about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.18%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4055
Misses ? 1313
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.48% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...6529692](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `57.34%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.26%
Complexity ? 964
=========================================
Files ? 104
Lines ? 5952
Branches ? 900
=========================================
Hits ? 4063
Misses ? 1310
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [...c/src/main/java/org/apache/livy/rsc/RSCClient.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDbGllbnQuamF2YQ==) | `73.49% <20%> (ø)` | `20 <1> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...fe5bfac](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384700532
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/LivyRestClient.scala
##########
@@ -75,14 +80,19 @@ class LivyRestClient(val httpClient: AsyncHttpClient, val livyEndpoint: String)
}
def snapshot(): SessionSnapshot = {
- val r = httpClient.prepareGet(url).execute().get()
- assertStatusCode(r, HttpServletResponse.SC_OK)
+ val httpGet = new HttpGet(url)
+ val r = httpClient.execute(httpGet)
+ assertStatusCode(r.getStatusLine(), HttpServletResponse.SC_OK)
- mapper.readValue(r.getResponseBodyAsStream, classOf[SessionSnapshot])
+ val sessionSnapshot = mapper.readValue(r.getEntity().getContent, classOf[SessionSnapshot])
+ r.close()
Review comment:
I now close the request before checking the assertions so this should be resolved
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583652698
@jerryshao requesting a review
The integration test seemed to have failed on a network connection issue how can I requeue the checks?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] ajbozarth commented on issue #278: Livy kerberos
and basic Auth for IT
Posted by GitBox <gi...@apache.org>.
ajbozarth commented on issue #278: Livy kerberos and basic Auth for IT
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-582034775
I believe @jerryshao would be a better reviewer for this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.18%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4055
Misses ? 1313
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.48% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...6529692](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384239252
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/conf/cluster.spec.template
##########
@@ -0,0 +1,46 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# type of cluster you are running integration tests against
Review comment:
Any reason to put this configuration in source code? I would think that this should be in `resources` or conf folder typically.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `60%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.23%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4058
Misses ? 1308
Partials ? 581
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...ain/java/org/apache/livy/rsc/driver/RSCDriver.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvUlNDRHJpdmVyLmphdmE=) | `81.66% <100%> (ø)` | `45 <0> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...3eeced5](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r384237207
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/BaseIntegrationTestSuite.scala
##########
@@ -96,7 +122,30 @@ abstract class BaseIntegrationTestSuite extends FunSuite with Matchers with Befo
// Please create an issue if this breaks test logging for cluster creation.
protected override def beforeAll() = {
cluster = Cluster.get()
- httpClient = new AsyncHttpClient()
+ httpClient = new DefaultHttpClient()
+
+ if (authScheme == "kerberos") {
+ val useJAASCreds = new Credentials() {
+ def getPassword(): String = {
+ return null
+ }
+
+ def getUserPrincipal(): Principal = {
+ return null
+ }
+ }
+
+ httpClient.getAuthSchemes().register(AuthPolicy.SPNEGO, new SPNegoSchemeFactory());
+ httpClient.getCredentialsProvider().setCredentials(
+ new AuthScope(null, -1, null),
+ useJAASCreds);
+ } else if (authScheme == "basic"){
Review comment:
nit: space after "(".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.19%
Complexity ? 964
=========================================
Files ? 104
Lines ? 5952
Branches ? 900
=========================================
Hits ? 4059
Misses ? 1314
Partials ? 579
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.03% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...e84310d](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-588556781
Thanks, I will review this soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] codecov-io edited a comment on issue #278:
[LIVY-748] Add support for running Livy Integration tests against secure
external clusters
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-583925153
# [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@07b2085`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `59.03%`.
[](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #278 +/- ##
=========================================
Coverage ? 68.13%
Complexity ? 960
=========================================
Files ? 104
Lines ? 5947
Branches ? 899
=========================================
Hits ? 4052
Misses ? 1314
Partials ? 581
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ain/java/org/apache/livy/rsc/driver/Statement.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9kcml2ZXIvU3RhdGVtZW50LmphdmE=) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...in/scala/org/apache/livy/server/ui/UIServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvdWkvVUlTZXJ2bGV0LnNjYWxh) | `4.49% <0%> (ø)` | `1 <0> (?)` | |
| [...a/org/apache/livy/utils/LivyUncaughtException.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS91dGlscy9MaXZ5VW5jYXVnaHRFeGNlcHRpb24uamF2YQ==) | `0% <0%> (ø)` | `0 <0> (?)` | |
| [...rver/src/main/scala/org/apache/livy/LivyConf.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9MaXZ5Q29uZi5zY2FsYQ==) | `96.13% <100%> (ø)` | `21 <0> (?)` | |
| [...che/livy/server/recovery/ZooKeeperStateStore.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvcmVjb3ZlcnkvWm9vS2VlcGVyU3RhdGVTdG9yZS5zY2FsYQ==) | `100% <100%> (ø)` | `8 <8> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/RSCConf.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9SU0NDb25mLmphdmE=) | `87.85% <100%> (ø)` | `8 <0> (?)` | |
| [rsc/src/main/java/org/apache/livy/rsc/Utils.java](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cnNjL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9saXZ5L3JzYy9VdGlscy5qYXZh) | `81.57% <100%> (ø)` | `14 <1> (?)` | |
| [...apache/livy/server/batch/BatchSessionServlet.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvYmF0Y2gvQmF0Y2hTZXNzaW9uU2VydmxldC5zY2FsYQ==) | `91.66% <100%> (ø)` | `3 <0> (?)` | |
| [.../src/main/scala/org/apache/livy/repl/Session.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-cmVwbC9zcmMvbWFpbi9zY2FsYS9vcmcvYXBhY2hlL2xpdnkvcmVwbC9TZXNzaW9uLnNjYWxh) | `67.69% <100%> (ø)` | `37 <0> (?)` | |
| [...main/scala/org/apache/livy/server/LivyServer.scala](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree#diff-c2VydmVyL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvbGl2eS9zZXJ2ZXIvTGl2eVNlcnZlci5zY2FsYQ==) | `33.03% <13.63%> (ø)` | `11 <0> (?)` | |
| ... and [14 more](https://codecov.io/gh/apache/incubator-livy/pull/278/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=footer). Last update [07b2085...26b1706](https://codecov.io/gh/apache/incubator-livy/pull/278?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao closed pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] RogPodge commented on issue #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
RogPodge commented on issue #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#issuecomment-585499899
@jerryshao requesting review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao closed pull request #278: [LIVY-748] Add
support for running Livy Integration tests against secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao closed pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-livy] jerryshao commented on a change in pull request
#278: [LIVY-748] Add support for running Livy Integration tests against
secure external clusters
Posted by GitBox <gi...@apache.org>.
jerryshao commented on a change in pull request #278: [LIVY-748] Add support for running Livy Integration tests against secure external clusters
URL: https://github.com/apache/incubator-livy/pull/278#discussion_r376837995
##########
File path: integration-test/src/main/scala/org/apache/livy/test/framework/Cluster.scala
##########
@@ -45,20 +47,46 @@ trait Cluster {
def jdbcEndpoint: Option[String]
def hdfsScratchDir(): Path
+ def authScheme: String
+ def user: String
+ def password: String
+ def sslCertPath: String
+
def doAsClusterUser[T](task: => T): T
- lazy val hadoopConf = {
+ def initKubeConf(): Configuration = {
val conf = new Configuration(false)
configDir().listFiles().foreach { f =>
if (f.getName().endsWith(".xml")) {
conf.addResource(new Path(f.toURI()))
}
}
+ UserGroupInformation.setConfiguration(conf);
+ UserGroupInformation.loginUserFromKeytab(s"${user}@AZDATA.LOCAL",
+ s"/tests/kerberos_setup/${user}.keytab");
Review comment:
Is it correct to hardcode principal and keytab here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services