You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/08/14 22:31:00 UTC
[jira] [Commented] (OOZIE-3036) Spark 2.2.0 support: tell Spark not
to get any delegation tokens
[ https://issues.apache.org/jira/browse/OOZIE-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126518#comment-16126518 ]
Hadoop QA commented on OOZIE-3036:
----------------------------------
Testing JIRA OOZIE-3036
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
. {color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 2009
. Tests rerun: 15
. Tests failed at first run: org.apache.oozie.jms.TestJMSJobEventListener,
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/4038/
> Spark 2.2.0 support: tell Spark not to get any delegation tokens
> ----------------------------------------------------------------
>
> Key: OOZIE-3036
> URL: https://issues.apache.org/jira/browse/OOZIE-3036
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.3.0
> Environment: * [*Spark 2.2.0*|https://github.com/apache/spark/tree/branch-2.2]
> * [*Hadoop 3 alpha 4*|https://github.com/apache/hadoop/tree/branch-3.0.0-alpha4]
> Reporter: Andras Piros
> Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-3036.001.patch
>
>
> For Spark 2.2.0 along w/ Hadoop 3 alpha 4, Spark uses another set of arguments that ensure Spark won't get any delegation tokens:
> * {{spark.yarn.security.credentials.hadoopfs.enabled}}
> * {{spark.yarn.security.credentials.hive.enabled}}
> * {{spark.yarn.security.credentials.hbase.enabled}}
> For Spark 1.6.0 along w/ Hadoop 3 alpha 4, we need following argument to set extra:
> * {{spark.yarn.security.tokens.hadoopfs.enabled}}
> As we can't tell runtime which Spark / Hadoop versions is the Oozie Spark action running on, it's best to use the union of those arguments.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)