You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "smaheshwar-pltr (via GitHub)" <gi...@apache.org> on 2024/12/23 16:02:28 UTC
[PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
smaheshwar-pltr opened a new pull request, #1466:
URL: https://github.com/apache/iceberg-python/pull/1466
This let me run integration tests locally. Before, I was getting
```
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. [...]
E : java.io.IOException: Failed to connect to [IP-REDACTED]
E at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:294)
E at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:214)
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #1466:
URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2569740575
I'm a bit torn on this one, it doesn't seem to cause any issues, but I'm inclined to think it is something with your local setup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on PR #1466:
URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2580707479
Thanks so much folks for confirming that my local setup was the problem - I looked more carefully and found `25/01/09 16:13:06 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address` which fixed it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr commented on PR #1466:
URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2559955607
Not sure if useful. @kevinjqliu, mind taking a peek?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #1466:
URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2560668984
@smaheshwar-pltr thanks for raising this. I haven't seen this before either. Can you check if your local hostname is configured correctly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #1466:
URL: https://github.com/apache/iceberg-python/pull/1466#issuecomment-2560159143
interesting, thats the first time ive seen this issue. do you have remote dev environment?
The typical set up is running the integration test docker containers on local laptop and pyspark will automatically find the driver.
That said i think explicitly binding to the `localhost` should still be fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Tests: Set PySpark driver host to `localhost` [iceberg-python]
Posted by "smaheshwar-pltr (via GitHub)" <gi...@apache.org>.
smaheshwar-pltr closed pull request #1466: Tests: Set PySpark driver host to `localhost`
URL: https://github.com/apache/iceberg-python/pull/1466
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org