You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by sjrand <gi...@git.apache.org> on 2017/10/20 04:52:56 UTC

[GitHub] spark pull request #19540: [SPARK-22319][Core] call loginUserFromKeytab befo...

GitHub user sjrand opened a pull request:

    https://github.com/apache/spark/pull/19540

    [SPARK-22319][Core] call loginUserFromKeytab before accessing hdfs

    ## What changes were proposed in this pull request?
    
    In `SparkSubmit`, call `loginUserFromKeytab` before attempting to make RPC calls to the NameNode.
    
    ## How was this patch tested?
    
    I manually tested this patch by:
    
    1. Confirming that my Spark application failed to launch with the error reported in https://issues.apache.org/jira/browse/SPARK-22319.
    2. Applying this patch and confirming that the app no longer fails to launch, even when I have not manually run `kinit` on the host.
    
    Presumably we also want integration tests for secure clusters so that we catch this sort of thing. I'm happy to take a shot at this if it's feasible and someone can point me in the right direction. 


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sjrand/spark SPARK-22319

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19540.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19540
    
----
commit 08240f368fdde6cf3ce02b50b7e7405a00f15fa4
Author: Steven Rand <sr...@palantir.com>
Date:   2017-10-19T04:51:26Z

    call loginUserFromKeytab before accessing hdfs

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by jerryshao <gi...@git.apache.org>.

Github user jerryshao commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    @sjrand , can you please create another PR against branch-2.2, it is not auto-mergeable, thanks!


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82927/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by jiangxb1987 <gi...@git.apache.org>.

Github user jiangxb1987 commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82954/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    **[Test build #82924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82924/testReport)** for PR 19540 at commit [`08240f3`](https://github.com/apache/spark/commit/08240f368fdde6cf3ce02b50b7e7405a00f15fa4).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by jerryshao <gi...@git.apache.org>.

Github user jerryshao commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    ok to test.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    **[Test build #82954 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82954/testReport)** for PR 19540 at commit [`08240f3`](https://github.com/apache/spark/commit/08240f368fdde6cf3ce02b50b7e7405a00f15fa4).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by jiangxb1987 <gi...@git.apache.org>.

Github user jiangxb1987 commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    Could you check why the test cases failed? Thanks!


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    **[Test build #82954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82954/testReport)** for PR 19540 at commit [`08240f3`](https://github.com/apache/spark/commit/08240f368fdde6cf3ce02b50b7e7405a00f15fa4).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #19540: [SPARK-22319][Core] call loginUserFromKeytab before acce...

Posted by jerryshao <gi...@git.apache.org>.

Github user jerryshao commented on the issue:

    https://github.com/apache/spark/pull/19540
  
    I think branch 2.2 also has similar issue when fetching resources from remote secure HDFS.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org