You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2017/04/03 22:14:41 UTC
[jira] [Commented] (ARROW-762) Kerberos Problem with PyArrow
[ https://issues.apache.org/jira/browse/ARROW-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954239#comment-15954239 ]
Wes McKinney commented on ARROW-762:
------------------------------------
Per our discussion in Slack, if you were able to resolve the problem, could you let me know what steps you took? I will write a patch to improve the documentation about this (or make the code itself more forgiving)
> Kerberos Problem with PyArrow
> -----------------------------
>
> Key: ARROW-762
> URL: https://issues.apache.org/jira/browse/ARROW-762
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 0.2.0
> Environment: Centos 7.2, HDP 2.4.3
> Reporter: Benjamin Zaitlen
>
> I'm having trouble using pyarrow with kerberos. I'm trying to connect to HDFS with the following signature:
> ```
> hdfs = HdfsClient(host='ip-172-31-53-87.ec2.internal', port=8020, kerb_ticket='/tmp/krb5cc_1000', driver='libhdfs3', user='centos')
> ArrowException Traceback (most recent call last)
> <ipython-input-2-15087f93c239> in <module>()
> ----> 1 hdfs = HdfsClient(host='ip-172-31-53-87.ec2.internal', port=8020, kerb_ticket='/tmp/krb5cc_1000', driver='libhdfs3', user='centos')
> /home/centos/miniconda3/envs/hdfs_test/lib/python3.5/site-packages/pyarrow/filesystem.py in __init__(self, host, port, user, kerb_ticket, driver)
> 168 def __init__(self, host="default", port=0, user=None, kerb_ticket=None,
> 169 driver='libhdfs'):
> --> 170 self._connect(host, port, user, kerb_ticket, driver)
> 171
> 172 @implements(Filesystem.isdir)
> /home/centos/miniconda3/envs/hdfs_test/lib/python3.5/site-packages/pyarrow/io.pyx in pyarrow.io._HdfsClient._connect (/feedstock_root/build_artefacts/pyarrow_1488727736041/work/arrow-f6924ad83bc95741f003830892ad4815ca3b70fd/python/build/temp.linux-x86_64-3.5/io.cxx:11090)()
> /home/centos/miniconda3/envs/hdfs_test/lib/python3.5/site-packages/pyarrow/error.pyx in pyarrow.error.check_status (/feedstock_root/build_artefacts/pyarrow_1488727736041/work/arrow-f6924ad83bc95741f003830892ad4815ca3b70fd/python/build/temp.linux-x86_64-3.5/error.cxx:1197)()
> ArrowException: IOError: HDFS connection failed
> ```
> Below shows a valid ticket:
> ```
> [centos@ip-172-31-61-224 usr]$ klist
> Ticket cache: FILE:/tmp/krb5cc_1000
> Default principal: centos@DOMAIN
> Valid starting Expires Service principal
> 04/03/2017 14:36:38 04/04/2017 14:36:38 krbtgt/DOMAIN@DOMAIN
> renew until 04/10/2017 14:36:38
> ```
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)