You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "farbodahm (via GitHub)" <gi...@apache.org> on 2024/09/17 09:21:40 UTC
[PR] feat: init devenv [spark-connect-go]
farbodahm opened a new pull request, #66:
URL: https://github.com/apache/spark-connect-go/pull/66
### What changes were proposed in this pull request?
This PR introduces [Devenv](https://devenv.sh/) to enhance the developer experience by providing a simpler and faster setup process for the development environment. With Devenv, setting up the development environment is now as simple as running:
```sh
devenv shell
sudo start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.5.1
```
This command will automatically launch a running Spark Connect instance.
(I will add more detailed documentation if the overall approach is accepted.)
### Why are the changes needed?
- Developers can easily set up identical development environments with the same versions across different operating systems.
- The setup process takes less than 2 minutes to complete, providing a fully functional development environment.
- All application versions (Java, Spark, etc.) are locked, ensuring consistent behavior across all environments.
- This setup is optional and does not interfere with manual installations, catering to developers who prefer an automated setup.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
If the approach is accepted, We can add build and test Github actions related to Devenv for automating testing the development environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] feat: init devenv [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip closed pull request #66: feat: init devenv
URL: https://github.com/apache/spark-connect-go/pull/66
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] feat: init devenv [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on PR #66:
URL: https://github.com/apache/spark-connect-go/pull/66#issuecomment-2367989539
Thanks for you contriutions! I have a couple of questions that mostly stem from my lack of familiarity of the devenv project.
* What is the intended usage pattern in conjunction with Spark Connect Go? Can you maybe document some typical developer tasks?
* Should starting Spark be modeled as a script or service - https://devenv.sh/scripts/
I'm trying to figure out how to help folks to get started with this concept! I think it could be a great improvement, we just need to figure out how to set it up most efficiently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] feat: init devenv [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on PR #66:
URL: https://github.com/apache/spark-connect-go/pull/66#issuecomment-3191535794
Closing due to inactivity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] feat: init devenv [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on code in PR #66:
URL: https://github.com/apache/spark-connect-go/pull/66#discussion_r1771260357
##########
devenv.nix:
##########
@@ -0,0 +1,43 @@
+{ pkgs, lib, config, inputs, ... }:
+
+{
+ # https://devenv.sh/basics/
+ env.GREET = "devenv";
+
+ # https://devenv.sh/packages/
+ packages = with pkgs; [
+ git
+ spark
Review Comment:
Can we pin it to a specific version?
##########
devenv.nix:
##########
@@ -0,0 +1,43 @@
+{ pkgs, lib, config, inputs, ... }:
+
+{
+ # https://devenv.sh/basics/
+ env.GREET = "devenv";
+
+ # https://devenv.sh/packages/
+ packages = with pkgs; [
Review Comment:
Should this add the Go dependency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] feat: init devenv [spark-connect-go]
Posted by "farbodahm (via GitHub)" <gi...@apache.org>.
farbodahm commented on PR #66:
URL: https://github.com/apache/spark-connect-go/pull/66#issuecomment-2368193413
@grundprinzip Thank you for the review, This week I'm on a vacation and will address the reviews as soon as I'm back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org