You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "luocan17 (via GitHub)" <gi...@apache.org> on 2024/02/20 06:36:26 UTC
[I] PyArrowFileIO.parse_location(...) return error path for hdfs location. [iceberg-python]
luocan17 opened a new issue, #449:
URL: https://github.com/apache/iceberg-python/issues/449
### Apache Iceberg version
None
### Please describe the bug 🐞
`parse_location` method in PyArrowFileIO return error `path` as follow.


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] `PyArrowFileIO.parse_location` return error `path` for hdfs location. [iceberg-python]
Posted by "syun64 (via GitHub)" <gi...@apache.org>.
syun64 commented on issue #449:
URL: https://github.com/apache/iceberg-python/issues/449#issuecomment-2083068661
Hi @luocan17 thank you for raising this issue. I have a PR up to attempt a fix. Could I ask for your review on: https://github.com/apache/iceberg-python/pull/668
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] `PyArrowFileIO.parse_location` return error `path` for hdfs location. [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #449: `PyArrowFileIO.parse_location` return error `path` for hdfs location.
URL: https://github.com/apache/iceberg-python/issues/449
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] `PyArrowFileIO.parse_location` return error `path` for hdfs location. [iceberg-python]
Posted by "j7nhai (via GitHub)" <gi...@apache.org>.
j7nhai commented on issue #449:
URL: https://github.com/apache/iceberg-python/issues/449#issuecomment-1966417221
how to fix this bug? I change 'location' to 'uri.path' but still get some errors.
(unable to get root cause for java.lang.NoClassDefFoundError)
(unable to get stack trace for java.lang.NoClassDefFoundError)
hdfsBuilderConnect(forceNewInstance=1, nn=hdfs://xxxxxx, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:
(unable to get root cause for java.lang.NoClassDefFoundError)
(unable to get stack trace for java.lang.NoClassDefFoundError)
/arrow/cpp/src/arrow/status.cc:155: Failed to disconnect hdfs client: IOError: HDFS hdfsFS::Disconnect failed. Detail: [errno 9] Bad file descriptor
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/j7nhai/third-party/iceberg-python/pyiceberg/catalog/hive.py", line 312, in create_table
self._write_metadata(metadata, io, metadata_location)
File "/home/j7nhai/third-party/iceberg-python/pyiceberg/catalog/__init__.py", line 651, in _write_metadata
ToOutputFile.table_metadata(metadata, io.new_output(metadata_path))
File "/home/j7nhai/third-party/iceberg-python/pyiceberg/io/pyarrow.py", line 422, in new_output
fs=self.fs_by_scheme(scheme, netloc),
File "/home/j7nhai/third-party/iceberg-python/pyiceberg/io/pyarrow.py", line 361, in _initialize_fs
return HadoopFileSystem.from_uri(f"{scheme}://{netloc}")
File "pyarrow/_hdfs.pyx", line 134, in pyarrow._hdfs.HadoopFileSystem.from_uri
File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
OSError: HDFS connection failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] `PyArrowFileIO.parse_location` return error `path` for hdfs location. [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #449:
URL: https://github.com/apache/iceberg-python/issues/449#issuecomment-2085249291
Fixed both by @syun64 and @frankliee
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org