You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "panbingkun (via GitHub)" <gi...@apache.org> on 2024/11/29 07:12:51 UTC
[PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
panbingkun opened a new pull request, #49013:
URL: https://github.com/apache/spark/pull/49013
### What changes were proposed in this pull request?
The pr aims to document `dev/spark-test-image-util/docs/build-docs` in `docs/README.md`.
### Why are the changes needed?
Add documentation for using script `dev/spark-test-image-util/docs/build-docs`.
### Does this PR introduce _any_ user-facing change?
Yes, Spark developers will be able to know the existence of script `dev/spark-test-image-util/docs/build-docs` through documentation `docs/README.md`.
### How was this patch tested?
Manually check.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863064771
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
+
+```sh
+sh dev/spark-test-image-util/docs/build-docs
+```
+
+It will generate all documents on the `container` and `host`.
+Especially when there are conflicts between the libraries required by python development environment
+and the libraries required by generating python docs environment, this is a good choice.
+Please refer to [SPARK-50295](https://issues.apache.org/jira/browse/SPARK-50295) for details.
Review Comment:
Okay
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
Review Comment:
Okay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863130489
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
git add --chmod=+x dev/spark-test-image-util/docs/build-docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863064142
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
+
+```sh
+sh dev/spark-test-image-util/docs/build-docs
+```
+
+It will generate all documents on the `container` and `host`.
+Especially when there are conflicts between the libraries required by python development environment
+and the libraries required by generating python docs environment, this is a good choice.
Review Comment:
```suggestion
and the libraries required by generating Python docs environment, this is a good choice.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863063991
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
+
+```sh
+sh dev/spark-test-image-util/docs/build-docs
+```
+
+It will generate all documents on the `container` and `host`.
+Especially when there are conflicts between the libraries required by python development environment
Review Comment:
```suggestion
Especially when there are conflicts between the libraries required by Python development environment
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #49013:
URL: https://github.com/apache/spark/pull/49013#issuecomment-2507217204
cc @HyukjinKwon @zhengruifeng @LuciferYang @pan3793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863101744
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
sh is not required after granting x permission
```suggestion
$ dev/spark-test-image-util/docs/build-docs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863064378
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
+
+```sh
+sh dev/spark-test-image-util/docs/build-docs
+```
+
+It will generate all documents on the `container` and `host`.
+Especially when there are conflicts between the libraries required by python development environment
+and the libraries required by generating python docs environment, this is a good choice.
+Please refer to [SPARK-50295](https://issues.apache.org/jira/browse/SPARK-50295) for details.
Review Comment:
I think no need to link the JIRA here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #49013:
URL: https://github.com/apache/spark/pull/49013#issuecomment-2507573549
Merged to master.
Thank you, @HyukjinKwon @zhengruifeng @pan3793 @LuciferYang .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863117655
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
But it's a bit strange, I granting it as `chmod a+x dev/spark-test-image-util/docs/build-docs`, but it didn't show up in the git logs. Do you know why? thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863117655
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
But it's a bit strange, I granting it as `chmod a+x dev/spark-test-image-util/docs/build-docs`, but it didn't show up in the git logs. Do you know why? thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #49013:
URL: https://github.com/apache/spark/pull/49013#issuecomment-2507576703
After the pr, the permissions for file `dev/spark-test-image-util/docs/build-docs` are correct.
```shell
(base) ➜ spark-trunk git:(master) ✗ ls -al dev/spark-test-image-util/docs/build-docs
-rwxr-xr-x@ 1 panbingkun staff 2794 Nov 29 18:59 dev/spark-test-image-util/docs/build-docs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863125285
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
It seems to work, let's take another look this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863064760
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
+As a Spark developer, you can generate all documents locally as follows:
Review Comment:
Let's mention Docker is required.
##########
docs/README.md:
##########
@@ -126,3 +126,15 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+
+## Build docs with docker image (Optional)
Review Comment:
```suggestion
## Build docs with docker image (Optional)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #49013:
URL: https://github.com/apache/spark/pull/49013#discussion_r1863120415
##########
docs/README.md:
##########
@@ -126,3 +126,16 @@ To control what API docs get built, you can set any combination of the following
* `SKIP_RDOC=1`: Skip the R API docs.
* `SKIP_SQLDOC=1`: Skip the SQL API docs.
+## Build docs with docker image (Optional)
+
+As a Spark developer, you can generate all documents locally as follows:
+
+Note: Before running it, you need to have `docker` installed.
+
+```sh
+$ sh dev/spark-test-image-util/docs/build-docs
Review Comment:
But it's a bit strange, I granting it as `chmod a+x dev/spark-test-image-util/docs/build-docs`, but it didn't show up in the git logs. Do you know why? thanks.
```
(base) ➜ spark-trunk git:(master) ✗ ls -al dev/spark-test-image-util/docs/build-docs
-rw-r--r--@ 1 panbingkun staff 2794 Nov 29 16:17 dev/spark-test-image-util/docs/build-docs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md` [spark]
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun closed pull request #49013: [SPARK-50295][DOCS][FOLLOWUP] Document `build-docs` in `docs/README.md`
URL: https://github.com/apache/spark/pull/49013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org