You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/12 13:08:43 UTC

[GitHub] [spark] WolverineJiang opened a new pull request, #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

WolverineJiang opened a new pull request, #39036:
URL: https://github.com/apache/spark/pull/39036

   ### What changes were proposed in this pull request?
   This PR unify the environment variable of *_PROTOC_EXEC_PATH to support that users can build and test `core`, `connect`, 'protobuf' module by use profile named `-Puser-defined-protoc` with specifying custom `protoc` executables.
   
   
   ### Why are the changes needed?
   As described in [SPARK-41485](https://issues.apache.org/jira/browse/SPARK-41485), at present, there are 3 similar environment variable of *_PROTOC_EXEC_PATH, but they use the same pb version. Because they are consistent in compilation, so I unify the environment variable names to simplify.
   
   
   ### Does this PR introduce _any_ user-facing change?
   No, the way to using official pre-release `protoc` binary files is activated by default.
   
   
   ### How was this patch tested?
   - Pass GitHub Actions
   - Manual test on CentOS6u3 and CentOS7u4
   ```bash
   export SPARK_PROTOC_EXEC_PATH=/path-to-protoc-exe
   ./build/mvn clean install -pl core -Puser-defined-protoc -am -DskipTests -DskipDefaultProtoc
   ./build/mvn clean install -pl connector/connect/common -Puser-defined-protoc -am -DskipTests
   ./build/mvn clean install -pl connector/protobuf -Puser-defined-protoc -am -DskipTests
   ./build/mvn clean test -pl core -Puser-defined-protoc -DskipDefaultProtoc
   ./build/mvn clean test -pl connector/connect/common -Puser-defined-protoc
   ./build/mvn clean test -pl connector/protobuf -Puser-defined-protoc
   ```
   and
   ```bash
   export SPARK_PROTOC_EXEC_PATH=/path-to-protoc-exe
   ./build/sbt clean "core/compile" -Puser-defined-protoc
   ./build/sbt clean "connect-common/compile" -Puser-defined-protoc
   ./build/sbt clean "protobuf/compile" -Puser-defined-protoc
   ./build/sbt "core/test" -Puser-defined-protoc
   ./build/sbt  "connect-common/test" -Puser-defined-protoc
   ./build/sbt  "protobuf/test" -Puser-defined-protoc
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] WolverineJiang commented on pull request #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

Posted by GitBox <gi...@apache.org>.
WolverineJiang commented on PR #39036:
URL: https://github.com/apache/spark/pull/39036#issuecomment-1347789334

   Thanks @HyukjinKwon @LuciferYang ~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on PR #39036:
URL: https://github.com/apache/spark/pull/39036#issuecomment-1347701826

   Merged to master.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] WolverineJiang commented on pull request #39036: [SPARK-41485][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

Posted by GitBox <gi...@apache.org>.
WolverineJiang commented on PR #39036:
URL: https://github.com/apache/spark/pull/39036#issuecomment-1386551587

   > The correct jira should be [SPARK-41485](https://issues.apache.org/jira/browse/SPARK-41485) rather than [SPARK-41461](https://issues.apache.org/jira/browse/SPARK-41461), please correct the pr title on this page, although the commit message cannot be changed. @WolverineJiang
   
   Done. I'll pay attention next time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] LuciferYang commented on pull request #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #39036:
URL: https://github.com/apache/spark/pull/39036#issuecomment-1386524523

   The correct jira should be SPARK-41485 rather than SPARK-41461,  please correct the pr title on this page, although the commit message cannot be changed. @WolverineJiang 
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon closed pull request #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.

Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #39036: [SPARK-41461][BUILD][CORE][CONNECT][PROTOBUF] Unify the environment variable of *_PROTOC_EXEC_PATH.
URL: https://github.com/apache/spark/pull/39036


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org