You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Alexey Klimov <kl...@megaputer.ru> on 2016/12/15 21:46:00 UTC
Is restarting of SparkContext allowed?
Hello, my question is the continuation of a problem I described here
<http://stackoverflow.com/questions/40834482/delegation-token-can-be-issued-only-with-kerberos-or-web-authentication-when-r>
.
I've done some investigation and found out that nameNode.getDelegationToken
is called during constructing SparkContext even if delegation token is
already presented in token list of current logged user in object of
UserGroupInforation class. The problem doesn't occur when waiting time
before constructing a new context is less than 10 seconds, because rpc
connection to namenode just isn't resetting because of
ipc.client.connection.maxidletime property.
As a workaround of this problem I do login from keytab before every
constructing of SparkContext, which basically just resets token list of
current logged user (as well as whole user structure) and the problem seems
to be gone. Still I'm not really sure that it is correct way to deal with
SparkContext.
Having found a reason of the problem, I've got 2 assumptions now:
First - SparkContext was designed to be restarted during JVM run and
behaviour above is just a bug.
Second - it wasn't and I'm just using SparkContext in a wrong manner.
Since I haven't found any related bug in Jira and any solution on the
internet (as well as too many users facing this error) I tend to think that
it is rather a not allowed usage of SparkContext.
Is that correct?
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Is-restarting-of-SparkContext-allowed-tp20240.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Is restarting of SparkContext allowed?
Posted by Alexey Klimov <kl...@megaputer.ru>.
I also wanted to ask that if it's not design way to use SparkContext how much
would it take to get it work completely correct. (e.g. are there any other
singletones that can preserve other state between running different
SparkContext's).
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Is-restarting-of-SparkContext-allowed-tp20240p20242.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Is restarting of SparkContext allowed?
Posted by Marcelo Vanzin <va...@cloudera.com>.
(-dev, +user. dev is for Spark development, not for questions about
using Spark.)
You haven't posted code here or the actual error. But you might be
running into SPARK-15754. Or into other issues with yarn-client mode
and "--principal / --keytab" (those have known issues in client mode).
If you have the above fix, you should be able to run the SparkContext
in client mode inside a UGI.doAs() block, after you login the user,
and later stop the context and start a new one. (And don't use
"--principal" / "--keytab" in that case.)
On Thu, Dec 15, 2016 at 1:46 PM, Alexey Klimov <kl...@megaputer.ru> wrote:
> Hello, my question is the continuation of a problem I described here
> <http://stackoverflow.com/questions/40834482/delegation-token-can-be-issued-only-with-kerberos-or-web-authentication-when-r>
> .
>
> I've done some investigation and found out that nameNode.getDelegationToken
> is called during constructing SparkContext even if delegation token is
> already presented in token list of current logged user in object of
> UserGroupInforation class. The problem doesn't occur when waiting time
> before constructing a new context is less than 10 seconds, because rpc
> connection to namenode just isn't resetting because of
> ipc.client.connection.maxidletime property.
>
> As a workaround of this problem I do login from keytab before every
> constructing of SparkContext, which basically just resets token list of
> current logged user (as well as whole user structure) and the problem seems
> to be gone. Still I'm not really sure that it is correct way to deal with
> SparkContext.
>
> Having found a reason of the problem, I've got 2 assumptions now:
> First - SparkContext was designed to be restarted during JVM run and
> behaviour above is just a bug.
> Second - it wasn't and I'm just using SparkContext in a wrong manner.
>
> Since I haven't found any related bug in Jira and any solution on the
> internet (as well as too many users facing this error) I tend to think that
> it is rather a not allowed usage of SparkContext.
>
> Is that correct?
>
>
>
> --
> View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Is-restarting-of-SparkContext-allowed-tp20240.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
--
Marcelo
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org