[jira] [Commented] (ARROW-2306) [Python] HDFS test failures

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398063#comment-16398063 ] 

ASF GitHub Bot commented on ARROW-2306:
---------------------------------------

wesm opened a new pull request #1742: ARROW-2306: [Python] Fix partitioned Parquet test against HDFS
URL: https://github.com/apache/arrow/pull/1742
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> [Python] HDFS test failures
> ---------------------------
>
>                 Key: ARROW-2306
>                 URL: https://issues.apache.org/jira/browse/ARROW-2306
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Wes McKinney
>            Assignee: Wes McKinney
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.9.0
>
>
> These weren't caught because we aren't running the HDFS tests in Travis CI
> {code}
> pyarrow/tests/test_hdfs.py::TestLibHdfs::test_write_to_dataset_no_partitions FAILED
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> self = <pyarrow.tests.test_hdfs.TestLibHdfs testMethod=test_write_to_dataset_no_partitions>
>     @test_parquet.parquet
>     def test_write_to_dataset_no_partitions(self):
>         tmpdir = pjoin(self.tmp_path, 'write-no_partitions-' + guid())
>         self.hdfs.mkdir(tmpdir)
>         test_parquet._test_write_to_dataset_no_partitions(
> >           tmpdir, filesystem=self.hdfs)
> pyarrow/tests/test_hdfs.py:367: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> pyarrow/tests/test_parquet.py:1475: in _test_write_to_dataset_no_partitions
>     filesystem=filesystem)
> pyarrow/parquet.py:1059: in write_to_dataset
>     _mkdir_if_not_exists(fs, root_path)
> pyarrow/parquet.py:1006: in _mkdir_if_not_exists
>     if fs._isfilestore() and not fs.exists(path):
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> self = <pyarrow.hdfs.HadoopFileSystem object at 0x7f09e87a4c48>
>     def _isfilestore(self):
>         """
>             Returns True if this FileSystem is a unix-style file store with
>             directories.
>             """
> >       raise NotImplementedError
> E       NotImplementedError
> pyarrow/filesystem.py:143: NotImplementedError
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > /home/wesm/code/arrow/python/pyarrow/filesystem.py(143)_isfilestore()
> -> raise NotImplementedError
> (Pdb) c
> pyarrow/tests/test_hdfs.py::TestLibHdfs::test_write_to_dataset_with_partitions FAILED
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> self = <pyarrow.tests.test_hdfs.TestLibHdfs testMethod=test_write_to_dataset_with_partitions>
>     @test_parquet.parquet
>     def test_write_to_dataset_with_partitions(self):
>         tmpdir = pjoin(self.tmp_path, 'write-partitions-' + guid())
>         self.hdfs.mkdir(tmpdir)
>         test_parquet._test_write_to_dataset_with_partitions(
> >           tmpdir, filesystem=self.hdfs)
> pyarrow/tests/test_hdfs.py:360: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> pyarrow/tests/test_parquet.py:1433: in _test_write_to_dataset_with_partitions
>     filesystem=filesystem)
> pyarrow/parquet.py:1059: in write_to_dataset
>     _mkdir_if_not_exists(fs, root_path)
> pyarrow/parquet.py:1006: in _mkdir_if_not_exists
>     if fs._isfilestore() and not fs.exists(path):
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> self = <pyarrow.hdfs.HadoopFileSystem object at 0x7f09e87a4c48>
>     def _isfilestore(self):
>         """
>             Returns True if this FileSystem is a unix-style file store with
>             directories.
>             """
> >       raise NotImplementedError
> E       NotImplementedError
> pyarrow/filesystem.py:143: NotImplementedError
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > /home/wesm/code/arrow/python/pyarrow/filesystem.py(143)_isfilestore()
> -> raise NotImplementedError
> (Pdb) c
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)