You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Dan Burkert (Code Review)" <ge...@cloudera.org> on 2017/05/10 00:24:46 UTC

[kudu-CR](branch-1.3.x) KUDU-1999: Spark connector should login with Kerberos credentials on driver 

Hello Jean-Daniel Cryans, Todd Lipcon, Kudu Jenkins,

I'd like you to do a code review.  Please visit

    http://gerrit.cloudera.org:8080/6836

to review the following change.

Change subject: KUDU-1999: Spark connector should login with Kerberos credentials on driver
......................................................................

KUDU-1999: Spark connector should login with Kerberos credentials on driver

Tested on a secure cluster using the Spark ITBLL job:

spark2-submit \
    --deploy-mode=cluster \
    --master=yarn \
    --principal=dan \
    --keytab dan.keytab \
    --class org.apache.kudu.spark.tools.IntegrationTestBigLinkedList \
    kudu-spark-tools-1.4.0-SNAPSHOT.jar generate \
    --master-addrs=kudu-spark-secure-1.gce.cloudera.com

Without some very major changes to our test infrastructure it's not
possible to test this code in unit tests, since it relies on a secure
Yarn cluster being available.

note: long-running jobs will continue to fail, since credentials are
still not refreshed.

Change-Id: If87a470c1cf99ea52668f22b72f1f7331877ec63
Reviewed-on: http://gerrit.cloudera.org:8080/6822
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <to...@apache.org>
---
M docs/developing.adoc
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
4 files changed, 79 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/6836/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6836
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If87a470c1cf99ea52668f22b72f1f7331877ec63
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR](branch-1.3.x) KUDU-1999: Spark connector should login with Kerberos credentials on driver

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.

Change subject: KUDU-1999: Spark connector should login with Kerberos credentials on driver
......................................................................


Patch Set 1: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/6836
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If87a470c1cf99ea52668f22b72f1f7331877ec63
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No

[kudu-CR](branch-1.3.x) KUDU-1999: Spark connector should login with Kerberos credentials on driver

Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has submitted this change and it was merged.

Change subject: KUDU-1999: Spark connector should login with Kerberos credentials on driver
......................................................................


KUDU-1999: Spark connector should login with Kerberos credentials on driver

Tested on a secure cluster using the Spark ITBLL job:

spark2-submit \
    --deploy-mode=cluster \
    --master=yarn \
    --principal=dan \
    --keytab dan.keytab \
    --class org.apache.kudu.spark.tools.IntegrationTestBigLinkedList \
    kudu-spark-tools-1.4.0-SNAPSHOT.jar generate \
    --master-addrs=kudu-spark-secure-1.gce.cloudera.com

Without some very major changes to our test infrastructure it's not
possible to test this code in unit tests, since it relies on a secure
Yarn cluster being available.

note: long-running jobs will continue to fail, since credentials are
still not refreshed.

Change-Id: If87a470c1cf99ea52668f22b72f1f7331877ec63
Reviewed-on: http://gerrit.cloudera.org:8080/6822
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <to...@apache.org>
Reviewed-on: http://gerrit.cloudera.org:8080/6836
---
M docs/developing.adoc
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/TestContext.scala
4 files changed, 79 insertions(+), 15 deletions(-)

Approvals:
  Todd Lipcon: Looks good to me, approved
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/6836
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If87a470c1cf99ea52668f22b72f1f7331877ec63
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: branch-1.3.x
Gerrit-Owner: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>