You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Gianmarco De Francisci Morales <gd...@apache.org> on 2012/06/20 16:37:33 UTC
Server is not configured to do DIGEST authentication
Hi,
I am getting the exception in the subject when running my giraph program on
a cluster with Kerberos authentication.
Here the stack trace:
2012-06-20 14:31:20,940 ERROR
org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:gdfm (auth:SIMPLE)
cause:org.apache.hadoop.ipc.RemoteException: Server is not configured
to do DIGEST authentication.
2012-06-20 14:31:20,940 WARN
org.apache.giraph.comm.BasicRPCCommunications: connectAllRPCProxys:
Failed on attempt 3 of 5 to connect to
(id=3,cur=Worker(hostname=gsta31997.tan.ygrid.yahoo.com,
MRpartition=1, port=30001),prev=null,ckpt_file=null)
org.apache.hadoop.ipc.RemoteException: Server is not configured to do
DIGEST authentication.
at org.apache.hadoop.ipc.Client.call(Client.java:1066)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy8.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:420)
at org.apache.giraph.comm.RPCCommunications.getRPCProxy(RPCCommunications.java:187)
at org.apache.giraph.comm.BasicRPCCommunications.startPeerConnectionThread(BasicRPCCommunications.java:684)
at org.apache.giraph.comm.BasicRPCCommunications.connectAllRPCProxys(BasicRPCCommunications.java:628)
at org.apache.giraph.comm.BasicRPCCommunications.setup(BasicRPCCommunications.java:589)
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:569)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:473)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:645)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
2012-06-20 14:31:20,941 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 30002: readAndProcess threw exception
org.apache.hadoop.security.AccessControlException: Server is not
configured to do DIGEST authentication.. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Server is not
configured to do DIGEST authentication.
at org.apache.hadoop.ipc.Server$Connection.saslReadAndProcess(Server.java:966)
at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1180)
at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:537)
at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:344)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
The result is:
2012-06-20 14:31:21,558 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.NullPointerException
at org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:911)
at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:322)
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:594)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:473)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:645)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
The same program works on my local hadoop instance (though the versions are
slightly different, 1.0.2 local and 0.20.205 on the cluster).
Is it a munging problem?
Thanks,
--
Gianmarco
Re: Server is not configured to do DIGEST authentication
Posted by Eugene Koontz <ek...@hiro-tan.org>.
On 6/21/12 11:48 AM, Jakob Homan wrote:
> I'm seeing the same problem on our secure clusters with the latest
> builds from trunk. I've opened GIRAPH-212 and am investigating...
>
Thanks Jakob,
I will look at this also starting tomorrow.
-Eugene
Re: Server is not configured to do DIGEST authentication
Posted by Jakob Homan <jg...@gmail.com>.
I'm seeing the same problem on our secure clusters with the latest
builds from trunk. I've opened GIRAPH-212 and am investigating...
On Thu, Jun 21, 2012 at 6:52 AM, Gianmarco De Francisci Morales
<gd...@apache.org> wrote:
> Thanks Eugene,
>
> just to answer your question on the configuration, I am unable to share the
> full configuration, and I don't directly administer the cluster.
> However, in core-site.xml we have:
>
> <name>hadoop.security.authentication</name>
> <value>kerberos</value>
>
> I don't know why it is trying to use digest as well (or what it exactly
> means).
>
> Cheers,
> --
> Gianmarco
>
>
>
>
> On Thu, Jun 21, 2012 at 12:20 AM, Eugene Koontz <ek...@hiro-tan.org>
> wrote:
>>
>> On 6/20/12 3:06 PM, Gianmarco De Francisci Morales wrote:
>> > Hi,
>> >
>> > Thanks for the advice on using Netty, it looks like it solved my
>> > problem.
>> >
>> > Now my job gets stuck at 99%, but it must be something different :)
>> >
>> > Cheers,
>> > --
>> > Gianmarco
>> >
>> >
>> >
>> I filed https://issues.apache.org/jira/browse/GIRAPH-211 regarding
>> authentication and Giraph. Thanks for sharing your experience, Gianmarco.
>>
>> -Eugene
>
>
Re: Server is not configured to do DIGEST authentication
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
Thanks Eugene,
just to answer your question on the configuration, I am unable to share the
full configuration, and I don't directly administer the cluster.
However, in core-site.xml we have:
<name>hadoop.security.authentication</name>
<value>kerberos</value>
I don't know why it is trying to use digest as well (or what it exactly
means).
Cheers,
--
Gianmarco
On Thu, Jun 21, 2012 at 12:20 AM, Eugene Koontz <ek...@hiro-tan.org>wrote:
> On 6/20/12 3:06 PM, Gianmarco De Francisci Morales wrote:
> > Hi,
> >
> > Thanks for the advice on using Netty, it looks like it solved my problem.
> >
> > Now my job gets stuck at 99%, but it must be something different :)
> >
> > Cheers,
> > --
> > Gianmarco
> >
> >
> >
> I filed https://issues.apache.org/jira/browse/GIRAPH-211 regarding
> authentication and Giraph. Thanks for sharing your experience, Gianmarco.
>
> -Eugene
>
Re: Server is not configured to do DIGEST authentication
Posted by Eugene Koontz <ek...@hiro-tan.org>.
On 6/20/12 3:06 PM, Gianmarco De Francisci Morales wrote:
> Hi,
>
> Thanks for the advice on using Netty, it looks like it solved my problem.
>
> Now my job gets stuck at 99%, but it must be something different :)
>
> Cheers,
> --
> Gianmarco
>
>
>
I filed https://issues.apache.org/jira/browse/GIRAPH-211 regarding
authentication and Giraph. Thanks for sharing your experience, Gianmarco.
-Eugene
Re: Server is not configured to do DIGEST authentication
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
Hi,
Thanks for the advice on using Netty, it looks like it solved my problem.
Now my job gets stuck at 99%, but it must be something different :)
Cheers,
--
Gianmarco
On Wed, Jun 20, 2012 at 8:06 PM, Avery Ching <ac...@apache.org> wrote:
> You can also try to use the netty option (no security).
>
> -Dgiraph.useNetty=true
>
> Avery
>
>
> On 6/20/12 10:57 AM, Eugene Koontz wrote:
>
>> On 6/20/12 7:37 AM, Gianmarco De Francisci Morales wrote:
>>
>>> Hi,
>>>
>>> I am getting the exception in the subject when running my giraph program
>>> on a cluster with Kerberos authentication.
>>>
>>> Here the stack trace:
>>>
>>> The same program works on my local hadoop instance (though the versions
>>> are slightly different, 1.0.2 local and 0.20.205 on the cluster).
>>> Is it a munging problem?
>>>
>>> Thanks,
>>> --
>>> Gianmarco
>>>
>>>
>>> Hi Gianmarco,
>> It's good to hear that you're trying out Giraph with
>> authentication. I
>> was a bit confused by your setup - are you using DIGEST or Kerberos? Can
>> you post your Hadoop configuration (your hadoop-site.xml)?
>>
>> -Eugene
>>
>>
>
Re: Server is not configured to do DIGEST authentication
Posted by Avery Ching <ac...@apache.org>.
You can also try to use the netty option (no security).
-Dgiraph.useNetty=true
Avery
On 6/20/12 10:57 AM, Eugene Koontz wrote:
> On 6/20/12 7:37 AM, Gianmarco De Francisci Morales wrote:
>> Hi,
>>
>> I am getting the exception in the subject when running my giraph program
>> on a cluster with Kerberos authentication.
>>
>> Here the stack trace:
>>
>> The same program works on my local hadoop instance (though the versions
>> are slightly different, 1.0.2 local and 0.20.205 on the cluster).
>> Is it a munging problem?
>>
>> Thanks,
>> --
>> Gianmarco
>>
>>
> Hi Gianmarco,
> It's good to hear that you're trying out Giraph with authentication. I
> was a bit confused by your setup - are you using DIGEST or Kerberos? Can
> you post your Hadoop configuration (your hadoop-site.xml)?
>
> -Eugene
>
Re: Server is not configured to do DIGEST authentication
Posted by Eugene Koontz <ek...@hiro-tan.org>.
On 6/20/12 7:37 AM, Gianmarco De Francisci Morales wrote:
> Hi,
>
> I am getting the exception in the subject when running my giraph program
> on a cluster with Kerberos authentication.
>
> Here the stack trace:
>
>
> The same program works on my local hadoop instance (though the versions
> are slightly different, 1.0.2 local and 0.20.205 on the cluster).
> Is it a munging problem?
>
> Thanks,
> --
> Gianmarco
>
>
Hi Gianmarco,
It's good to hear that you're trying out Giraph with authentication. I
was a bit confused by your setup - are you using DIGEST or Kerberos? Can
you post your Hadoop configuration (your hadoop-site.xml)?
-Eugene