You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/06/17 13:09:11 UTC
[GitHub] [arrow] jorisvandenbossche opened a new pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
jorisvandenbossche opened a new pull request #7469:
URL: https://github.com/apache/arrow/pull/7469
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645364205
@pitrou what would you think of something like this?
I saw several times users being confused about the typical AttributeError (eg https://github.com/apache/arrow/issues/7443, https://issues.apache.org/jira/browse/ARROW-8832).
This way, we could provide a more informative error message when pyarrow is not built with support for the specific filesystem (the error message in the PR can be further improved then)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645367030
https://issues.apache.org/jira/browse/ARROW-8832
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #7469:
URL: https://github.com/apache/arrow/pull/7469
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645924215
Ah, yes ;) Certainly still useful then.
Are you fine with not testing this on CI? (I tested it locally by disabling S3, and in the end it is only a different error message).
Although writing a test that is skipped when S3 is available is also not that hard, actually.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645916538
Updated.
Of course, now you added S3 support in wheels, this change will be less needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645916830
Only Linux wheels ;-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#issuecomment-645927818
Yes, it's ok if it's not tested, IMHO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7469: ARROW-8832: [Python] Provide better error message when S3/HDFS is not enabled in installation
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7469:
URL: https://github.com/apache/arrow/pull/7469#discussion_r441596426
##########
File path: python/pyarrow/fs.py
##########
@@ -35,14 +36,31 @@
# For backward compatibility.
FileStats = FileInfo
+_not_imported = []
+
try:
from pyarrow._hdfs import HadoopFileSystem # noqa
except ImportError:
- pass
+ _not_imported.append("HadoopFileSystem")
try:
from pyarrow._s3fs import S3FileSystem, initialize_s3, finalize_s3 # noqa
except ImportError:
- pass
+ _not_imported.append("S3FileSystem")
else:
initialize_s3()
+
+
+if sys.version_info >= (3, 7):
Review comment:
I don't think this is required. At worse `__getattr__` is a no-op :-)
##########
File path: python/pyarrow/fs.py
##########
@@ -35,14 +36,31 @@
# For backward compatibility.
FileStats = FileInfo
+_not_imported = []
+
try:
from pyarrow._hdfs import HadoopFileSystem # noqa
except ImportError:
- pass
+ _not_imported.append("HadoopFileSystem")
try:
from pyarrow._s3fs import S3FileSystem, initialize_s3, finalize_s3 # noqa
except ImportError:
- pass
+ _not_imported.append("S3FileSystem")
else:
initialize_s3()
+
+
+if sys.version_info >= (3, 7):
+
+ def __getattr__(name):
+
Review comment:
Spurious empty line?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org