You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2018/12/18 02:01:00 UTC
[jira] [Commented] (ARROW-4053) [Python/Integration] HDFS Tests
failing with I/O operation on closed file
[ https://issues.apache.org/jira/browse/ARROW-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723584#comment-16723584 ]
Wes McKinney commented on ARROW-4053:
-------------------------------------
It seems like the {{closed}} attribute was not added to HdfsFile
> [Python/Integration] HDFS Tests failing with I/O operation on closed file
> -------------------------------------------------------------------------
>
> Key: ARROW-4053
> URL: https://issues.apache.org/jira/browse/ARROW-4053
> Project: Apache Arrow
> Issue Type: Bug
> Components: Integration, Python
> Reporter: Krisztian Szucs
> Assignee: Wes McKinney
> Priority: Major
> Labels: hdfs
>
> To reporoduce run `docker-compose run hdfs-integration`
> {code:python}
> self = <pyarrow.tests.test_hdfs.TestLibHdfs testMethod=test_cat>
> def test_cat(self):
> path = pjoin(self.tmp_path, 'cat-test')
> data = b'foobarbaz'
> with self.hdfs.open(path, 'wb') as f:
> f.write(data)
> > contents = self.hdfs.cat(path)
> opt/conda/lib/python3.6/site-packages/pyarrow/tests/test_hdfs.py:99:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> opt/conda/lib/python3.6/site-packages/pyarrow/filesystem.py:42: in cat
> return f.read()
> pyarrow/io.pxi:298: in pyarrow.lib.NativeFile.read
> ???
> pyarrow/io.pxi:180: in pyarrow.lib.NativeFile.size
> ???
> pyarrow/io.pxi:142: in pyarrow.lib.NativeFile.get_random_access_file
> ???
> pyarrow/io.pxi:159: in pyarrow.lib.NativeFile._assert_readable
> ???
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> > ???
> E ValueError: I/O operation on closed file
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)