You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "grundprinzip (via GitHub)" <gi...@apache.org> on 2024/02/13 13:00:45 UTC
[PR] [MINOR][CONNECT] Allow Spark Connect Server Script to wait [spark]
grundprinzip opened a new pull request, #45090:
URL: https://github.com/apache/spark/pull/45090
### What changes were proposed in this pull request?
Add an option to the command line of `./sbin/start-connect-server.sh` that leaves it running in the foreground for easier debugging.
```
./sbin/start-connect-server.sh --wait
```
### Why are the changes needed?
Usability
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
Manual
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1945309543
> ... that leaves it running in the foreground ...
could it be achieved by the following command?
```
SPARK_NO_DAEMONIZE=1 ./sbin/start-connect-server.sh
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1945326307
Oh, right. Does it work in the same with this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1954587487
@pan3793 's point makes sense to me in terms of the consistency inside Apache Spark.
How do you think about that, @grundprinzip ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1945326854
If then, we need to revert this. Could you confirm that the above works for you, @grundprinzip ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #45090: [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait
URL: https://github.com/apache/spark/pull/45090
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1945634442
> If then, we need to revert this. Could you confirm that the above works for you, @grundprinzip ?
I will not revert the patch because it has significantly better developer ergonomics over obscure environment variables. I created a PR to use the env variable instead of calling spark-submit directly.
https://github.com/apache/spark/pull/45117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47040][CONNECT] Allow Spark Connect Server Script to wait [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #45090:
URL: https://github.com/apache/spark/pull/45090#issuecomment-1953579857
> ... it has significantly better developer ergonomics over obscure environment variables.
I would recommend removing `--wait`, just use env var and update https://spark.apache.org/developer-tools.html to mention that.
@grundprinzip There are several server components in Spark, like Thrift Server, History Server, Master, Worker, only adding `--wait` on the Connector Server makes things inconsistent, while env vars approach is adopted widely in Spark for different developer/testing purposes. Additionally, `--wait` is not quite accurate, it also affects stdout/stderr redirection not just process exiting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org