You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/09 14:18:30 UTC
[GitHub] [arrow] jorisvandenbossche opened a new pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
jorisvandenbossche opened a new pull request #8413:
URL: https://github.com/apache/arrow/pull/8413
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#discussion_r502763260
##########
File path: python/pyarrow/tests/test_hdfs.py
##########
@@ -321,7 +321,8 @@ def test_read_multiple_parquet_files_with_uri(self):
expected = self._write_multiple_hdfs_pq_files(tmpdir)
path = _get_hdfs_uri(tmpdir)
- result = pq.read_table(path)
+ # TODO for URI it should not be needed to pass this argument
+ result = pq.read_table(path, use_legacy_dataset=True)
Review comment:
Yes, was planning to -> https://issues.apache.org/jira/browse/ARROW-10264
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#discussion_r502441529
##########
File path: python/pyarrow/tests/test_hdfs.py
##########
@@ -321,7 +321,8 @@ def test_read_multiple_parquet_files_with_uri(self):
expected = self._write_multiple_hdfs_pq_files(tmpdir)
path = _get_hdfs_uri(tmpdir)
- result = pq.read_table(path)
+ # TODO for URI it should not be needed to pass this argument
+ result = pq.read_table(path, use_legacy_dataset=True)
Review comment:
This _should_ not be necessary, I would think, since reading from a URI should work with the new datasets implementation as well.
The error is `pyarrow.lib.ArrowInvalid: Path '/tmp/pyarrow-test-838/multi-parquet-uri-48569714efc74397816722c9c6723191/0.parquet' is not relative to '/user/root'`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#issuecomment-706193657
@github-actions crossbow submit test-conda-python-3.7-hdfs-2.9.2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs closed pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
kszucs closed pull request #8413:
URL: https://github.com/apache/arrow/pull/8413
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#issuecomment-706219230
HDFS integration build is green now ;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#issuecomment-706194766
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs commented on a change in pull request #8413: ARROW-10175: [CI] Fix nightly HDFS integration tests (ensure to use legacy dataset)
Posted by GitBox <gi...@apache.org>.
kszucs commented on a change in pull request #8413:
URL: https://github.com/apache/arrow/pull/8413#discussion_r502506815
##########
File path: python/pyarrow/tests/test_hdfs.py
##########
@@ -321,7 +321,8 @@ def test_read_multiple_parquet_files_with_uri(self):
expected = self._write_multiple_hdfs_pq_files(tmpdir)
path = _get_hdfs_uri(tmpdir)
- result = pq.read_table(path)
+ # TODO for URI it should not be needed to pass this argument
+ result = pq.read_table(path, use_legacy_dataset=True)
Review comment:
Perhaps create a jira?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org