You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "MirerRon (via GitHub)" <gi...@apache.org> on 2025/04/14 07:09:37 UTC
[I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
MirerRon opened a new issue, #12791:
URL: https://github.com/apache/iceberg/issues/12791
### Query engine
Hi everyone!
I'm trying the apache/iceberg-rest-fixture image with the org.apache.iceberg.hadoop.HadoopFileIO implementation and JDBC catalog type. I've compiled the program with the required hadoop dependencies, and it seems to work on simple 1 namenode cluster. But as soon as I moved to an HA enabled cluster I get the following error:
java.net.UnknownHostExcepption: Nameservice1
I have added the CATALOG_HADOOP_CONF_DIR env variable, and made sure to put the hdfs-site.xml and core-site.xml files in the directory. Though, it seems to be ignoring the hdfs-site.xml configuration, as that's where the nameservice and HA configurations are detailed.
Couldn't find anything concrete about only loading the core-site.xml file which could explain this issue. Did anyone encounter something similiar?
### Question
Is there any possibility of the HadoopCatalogIO implementation only loading the core-site.xml config file and not the hdfs-site.xml config file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #12791: iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException
URL: https://github.com/apache/iceberg/issues/12791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on issue #12791:
URL: https://github.com/apache/iceberg/issues/12791#issuecomment-2800717890
Thanks for raising the question here! Can you confirm that the files are part of the Docker image itself?
Just as a heads-up, the `iceberg-rest-fixture` container is meant for testing purposes and shouldn't be used in production environments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #12791:
URL: https://github.com/apache/iceberg/issues/12791#issuecomment-3393760032
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
Posted by "MirerRon (via GitHub)" <gi...@apache.org>.
MirerRon commented on issue #12791:
URL: https://github.com/apache/iceberg/issues/12791#issuecomment-2800725439
Can confirm (I'm mounting them as a volume but they do appear in runtime). The thing is, if I disable HA on my HDFS cluster then everything works as expected so I assume it is able to read the core-site.xml file.
Yes I'm aware of it, thanks for the heads up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] iceberg-rest-fixture Can't resolve HDFS HA Nameservice - UnknownHostException [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #12791:
URL: https://github.com/apache/iceberg/issues/12791#issuecomment-3447887612
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org