You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Michele Lamarca <mi...@brightcomputing.com> on 2017/08/18 08:37:54 UTC
Unable to SELECT from parquet file with Hadoop 2.7.4
Hi all,
I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following
select fails with 2.7.4, while it correctly executes with 2.7.3. I
found a possible relevant change here
https://issues.apache.org/jira/browse/HDFS-10673
Below you can find the error I get with 2.7.4.
Thanks in advance!
Best regards,
Michele
1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is
not a directory)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
Re: Unable to SELECT from parquet file with Hadoop 2.7.4
Posted by Michele Lamarca <mi...@brightcomputing.com>.
Hi Kunal,
Just filed https://issues.apache.org/jira/browse/DRILL-5733
Kind regards,
Michele
On Fri, Aug 18, 2017 at 10:35 PM, Kunal Khatua <kk...@mapr.com> wrote:
> Hi Michele
>
> Could you also share the entire stack track in the Drillbit log when logging the JIRA?
>
> I think you might have run this through SQLLine, so the entire stack trace might not be showing up. It helps to know where in the Drill code was this stack trace's call originating.
>
> Thanks
> Kunal
>
>
>
> -----Original Message-----
> From: Kunal Khatua [mailto:kkhatua@mapr.com]
> Sent: Friday, August 18, 2017 1:07 PM
> To: user@drill.apache.org
> Subject: RE: Unable to SELECT from parquet file with Hadoop 2.7.4
>
> Glad that worked !
>
> Not sure why the APIs in 2.7.4+ are not backward compatible, but it needs to be looked into.
>
> Could you file a JIRA ( https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185&projectKey=DRILL ) with these details that you've shared?
>
> That helps us identify and prioritize issues faced by the community and allocate resources for it.
>
> Thanks
> Kunal
>
> -----Original Message-----
> From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
> Sent: Friday, August 18, 2017 12:13 PM
> To: user@drill.apache.org
> Subject: Re: Unable to SELECT from parquet file with Hadoop 2.7.4
>
> Hi Kunal,
>
> That does the trick! I moved nation.parquet to /user/drill/test, then executed
>
> SELECT * FROM hdfs.`/user/drill/test`;
>
> without errors. Thanks a lot for the suggestion!
>
> For completeness, I was able to reproduce the error with:
> Hadoop 2.7.4 with Drill 1.11 (default pom.xml) Hadoop 2.7.4 with Drill 1.11 (with -Dhadoop.version=2.7.4) Hadoop 2.8.0 with Drill 1.11 (default pom.xml) Hadoop 3.0.0-alpha4 with Drill 1.11 (default pom.xml)
>
> Kind regards,
> Michele
>
>
> On Fri, Aug 18, 2017 at 8:06 PM, Kunal Khatua <kk...@mapr.com> wrote:
>> Interesting. I'm presuming this works if the parquet file is in a directory, right?
>> Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ?
>>
>> A workaround for now would be to query on an enclosing directory, until someone looks at the issue in more detail.
>>
>> -----Original Message-----
>> From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
>> Sent: Friday, August 18, 2017 1:38 AM
>> To: user@drill.apache.org
>> Subject: Unable to SELECT from parquet file with Hadoop 2.7.4
>>
>> Hi all,
>>
>> I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following
>> select fails with 2.7.4, while it correctly executes with 2.7.3. I
>> found a possible relevant change here
>> https://issues.apache.org/jira/browse/HDFS-10673
>>
>> Below you can find the error I get with 2.7.4.
>>
>> Thanks in advance!
>>
>> Best regards,
>> Michele
>>
>>
>>
>> 1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
>> Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is not a directory)
>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
>> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
>> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
>> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
>> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
>> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
>> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
RE: Unable to SELECT from parquet file with Hadoop 2.7.4
Posted by Kunal Khatua <kk...@mapr.com>.
Hi Michele
Could you also share the entire stack track in the Drillbit log when logging the JIRA?
I think you might have run this through SQLLine, so the entire stack trace might not be showing up. It helps to know where in the Drill code was this stack trace's call originating.
Thanks
Kunal
-----Original Message-----
From: Kunal Khatua [mailto:kkhatua@mapr.com]
Sent: Friday, August 18, 2017 1:07 PM
To: user@drill.apache.org
Subject: RE: Unable to SELECT from parquet file with Hadoop 2.7.4
Glad that worked !
Not sure why the APIs in 2.7.4+ are not backward compatible, but it needs to be looked into.
Could you file a JIRA ( https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185&projectKey=DRILL ) with these details that you've shared?
That helps us identify and prioritize issues faced by the community and allocate resources for it.
Thanks
Kunal
-----Original Message-----
From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
Sent: Friday, August 18, 2017 12:13 PM
To: user@drill.apache.org
Subject: Re: Unable to SELECT from parquet file with Hadoop 2.7.4
Hi Kunal,
That does the trick! I moved nation.parquet to /user/drill/test, then executed
SELECT * FROM hdfs.`/user/drill/test`;
without errors. Thanks a lot for the suggestion!
For completeness, I was able to reproduce the error with:
Hadoop 2.7.4 with Drill 1.11 (default pom.xml) Hadoop 2.7.4 with Drill 1.11 (with -Dhadoop.version=2.7.4) Hadoop 2.8.0 with Drill 1.11 (default pom.xml) Hadoop 3.0.0-alpha4 with Drill 1.11 (default pom.xml)
Kind regards,
Michele
On Fri, Aug 18, 2017 at 8:06 PM, Kunal Khatua <kk...@mapr.com> wrote:
> Interesting. I'm presuming this works if the parquet file is in a directory, right?
> Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ?
>
> A workaround for now would be to query on an enclosing directory, until someone looks at the issue in more detail.
>
> -----Original Message-----
> From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
> Sent: Friday, August 18, 2017 1:38 AM
> To: user@drill.apache.org
> Subject: Unable to SELECT from parquet file with Hadoop 2.7.4
>
> Hi all,
>
> I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following
> select fails with 2.7.4, while it correctly executes with 2.7.3. I
> found a possible relevant change here
> https://issues.apache.org/jira/browse/HDFS-10673
>
> Below you can find the error I get with 2.7.4.
>
> Thanks in advance!
>
> Best regards,
> Michele
>
>
>
> 1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
> Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is not a directory)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
RE: Unable to SELECT from parquet file with Hadoop 2.7.4
Posted by Kunal Khatua <kk...@mapr.com>.
Glad that worked !
Not sure why the APIs in 2.7.4+ are not backward compatible, but it needs to be looked into.
Could you file a JIRA ( https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185&projectKey=DRILL ) with these details that you've shared?
That helps us identify and prioritize issues faced by the community and allocate resources for it.
Thanks
Kunal
-----Original Message-----
From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
Sent: Friday, August 18, 2017 12:13 PM
To: user@drill.apache.org
Subject: Re: Unable to SELECT from parquet file with Hadoop 2.7.4
Hi Kunal,
That does the trick! I moved nation.parquet to /user/drill/test, then executed
SELECT * FROM hdfs.`/user/drill/test`;
without errors. Thanks a lot for the suggestion!
For completeness, I was able to reproduce the error with:
Hadoop 2.7.4 with Drill 1.11 (default pom.xml) Hadoop 2.7.4 with Drill 1.11 (with -Dhadoop.version=2.7.4) Hadoop 2.8.0 with Drill 1.11 (default pom.xml) Hadoop 3.0.0-alpha4 with Drill 1.11 (default pom.xml)
Kind regards,
Michele
On Fri, Aug 18, 2017 at 8:06 PM, Kunal Khatua <kk...@mapr.com> wrote:
> Interesting. I'm presuming this works if the parquet file is in a directory, right?
> Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ?
>
> A workaround for now would be to query on an enclosing directory, until someone looks at the issue in more detail.
>
> -----Original Message-----
> From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
> Sent: Friday, August 18, 2017 1:38 AM
> To: user@drill.apache.org
> Subject: Unable to SELECT from parquet file with Hadoop 2.7.4
>
> Hi all,
>
> I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following
> select fails with 2.7.4, while it correctly executes with 2.7.3. I
> found a possible relevant change here
> https://issues.apache.org/jira/browse/HDFS-10673
>
> Below you can find the error I get with 2.7.4.
>
> Thanks in advance!
>
> Best regards,
> Michele
>
>
>
> 1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
> Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is not a directory)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
Re: Unable to SELECT from parquet file with Hadoop 2.7.4
Posted by Michele Lamarca <mi...@brightcomputing.com>.
Hi Kunal,
That does the trick! I moved nation.parquet to /user/drill/test, then executed
SELECT * FROM hdfs.`/user/drill/test`;
without errors. Thanks a lot for the suggestion!
For completeness, I was able to reproduce the error with:
Hadoop 2.7.4 with Drill 1.11 (default pom.xml)
Hadoop 2.7.4 with Drill 1.11 (with -Dhadoop.version=2.7.4)
Hadoop 2.8.0 with Drill 1.11 (default pom.xml)
Hadoop 3.0.0-alpha4 with Drill 1.11 (default pom.xml)
Kind regards,
Michele
On Fri, Aug 18, 2017 at 8:06 PM, Kunal Khatua <kk...@mapr.com> wrote:
> Interesting. I'm presuming this works if the parquet file is in a directory, right?
> Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ?
>
> A workaround for now would be to query on an enclosing directory, until someone looks at the issue in more detail.
>
> -----Original Message-----
> From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
> Sent: Friday, August 18, 2017 1:38 AM
> To: user@drill.apache.org
> Subject: Unable to SELECT from parquet file with Hadoop 2.7.4
>
> Hi all,
>
> I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following select fails with 2.7.4, while it correctly executes with 2.7.3. I found a possible relevant change here
> https://issues.apache.org/jira/browse/HDFS-10673
>
> Below you can find the error I get with 2.7.4.
>
> Thanks in advance!
>
> Best regards,
> Michele
>
>
>
> 1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
> Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is not a directory)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
> at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
> at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
> at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
RE: Unable to SELECT from parquet file with Hadoop 2.7.4
Posted by Kunal Khatua <kk...@mapr.com>.
Interesting. I'm presuming this works if the parquet file is in a directory, right?
Was Drill built with Hadoop 2.7.4 dependencies or did you use the default 2.7.1 that is there in the POM.XML ?
A workaround for now would be to query on an enclosing directory, until someone looks at the issue in more detail.
-----Original Message-----
From: Michele Lamarca [mailto:michele.lamarca@brightcomputing.com]
Sent: Friday, August 18, 2017 1:38 AM
To: user@drill.apache.org
Subject: Unable to SELECT from parquet file with Hadoop 2.7.4
Hi all,
I'm currently testing Drill 1.11.0 with Hadoop 2.7.4. The following select fails with 2.7.4, while it correctly executes with 2.7.3. I found a possible relevant change here
https://issues.apache.org/jira/browse/HDFS-10673
Below you can find the error I get with 2.7.4.
Thanks in advance!
Best regards,
Michele
1/2 SELECT * FROM hdfs.`/user/drill/nation.parquet`;
Error: SYSTEM ERROR: RemoteException: /user/drill/nation.parquet (is not a directory)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkTraverse(FSPermissionChecker.java:272)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:215)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:199)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3820)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)