Re: [I] `PyArrowFileIO.parse_location` return error `path` for hdfs location. [iceberg-python]

["j7nhai (via GitHub)" <gi...@apache.org>]

j7nhai commented on issue #449:
URL: https://github.com/apache/iceberg-python/issues/449#issuecomment-1966417221

   how to fix this bug? I change 'location' to 'uri.path' but still get some errors.
   (unable to get root cause for java.lang.NoClassDefFoundError)
   (unable to get stack trace for java.lang.NoClassDefFoundError)
   hdfsBuilderConnect(forceNewInstance=1, nn=hdfs://xxxxxx, port=0, kerbTicketCachePath=(NULL), userName=(NULL)) error:
   (unable to get root cause for java.lang.NoClassDefFoundError)
   (unable to get stack trace for java.lang.NoClassDefFoundError)
   /arrow/cpp/src/arrow/status.cc:155: Failed to disconnect hdfs client: IOError: HDFS hdfsFS::Disconnect failed. Detail: [errno 9] Bad file descriptor
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/home/j7nhai/third-party/iceberg-python/pyiceberg/catalog/hive.py", line 312, in create_table
       self._write_metadata(metadata, io, metadata_location)
     File "/home/j7nhai/third-party/iceberg-python/pyiceberg/catalog/__init__.py", line 651, in _write_metadata
       ToOutputFile.table_metadata(metadata, io.new_output(metadata_path))
     File "/home/j7nhai/third-party/iceberg-python/pyiceberg/io/pyarrow.py", line 422, in new_output
       fs=self.fs_by_scheme(scheme, netloc),
     File "/home/j7nhai/third-party/iceberg-python/pyiceberg/io/pyarrow.py", line 361, in _initialize_fs
       return HadoopFileSystem.from_uri(f"{scheme}://{netloc}")
     File "pyarrow/_hdfs.pyx", line 134, in pyarrow._hdfs.HadoopFileSystem.from_uri
     File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
     File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
   OSError: HDFS connection failed


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org