You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2023/07/17 01:05:39 UTC
[GitHub] [spark] zhengruifeng opened a new pull request, #42028: [SPARK-44451][BUILD] Make built document downloadable
zhengruifeng opened a new pull request, #42028:
URL: https://github.com/apache/spark/pull/42028
### What changes were proposed in this pull request?
Make built document downloadable
### Why are the changes needed?
1, to make it more easier for reviewers to verify the document PRs;
2, to make it more easier for maintainers to check the built document in branches;
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
CI and manually check the downloaded docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264790680
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
+ with:
+ name: site
Review Comment:
good point, I am not aware about the quota limited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264820297
##########
dev/merge_spark_pr.py:
##########
@@ -44,7 +44,7 @@
# Location of your Spark git development area
SPARK_HOME = os.environ.get("SPARK_HOME", os.getcwd())
-# Remote name which points to the Gihub site
+# Remote name which points to the Github site
Review Comment:
This is correct, but should it be in an independent pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #42028:
URL: https://github.com/apache/spark/pull/42028#issuecomment-1639677885
thank you all
merged to master (4.0.0)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264782031
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
Review Comment:
Does this help? I thought there are separate workspace context in every step. So it seems could be removed.
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
Review Comment:
```
if: github.repository != 'apache/spark'
```
I am not sure do we want to add a condition in these two step like above to skip the upload progress in apache/repo (work in PR but not post merge commits, the PR trigger the job in personal repo rather than apache repo)?
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
+ with:
+ name: site
Review Comment:
```
retention-period: 1
```
As I mentioned before, I recommand to set `retention-period` to `1` to avoid potential artifact storage quota limited.
Background: I encounetered the issue like `Cannot upload artifacts. You may have exceeded your spending limit for storing artifacts with Actions or Packages.` [2], but there still no expilict description in github official doc to show the real limited quota [3].
[1] https://github.com/actions/upload-artifact#retention-period
[2] https://github.com/actions/upload-artifact/issues/307
[3] https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts#about-workflow-artifacts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264790443
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
Review Comment:
that depends on whether we need to monitor the build-doc in apache/repo.
I personally want to also upload in apache/repo, since the cost seems quietly small.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264897209
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
Review Comment:
on second thought, I think we can add this condition first, to be more conservative
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264790209
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
Review Comment:
Yeah, I think we don't need this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng closed pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
URL: https://github.com/apache/spark/pull/42028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #42028:
URL: https://github.com/apache/spark/pull/42028#issuecomment-1637256382
https://github.com/zhengruifeng/spark/actions/runs/5566016597/jobs/10167115410
I also test directly upload the directory, but it is slower and the artifact size is bigger (~112M):
(actually 112M to download)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Yikun commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "Yikun (via GitHub)" <gi...@apache.org>.
Yikun commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264784797
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
+ with:
+ name: site
Review Comment:
```
retention-days: 1
```
As I mentioned before, I recommand to set `retention-period` to `1` to avoid potential artifact storage quota limited.
Background: I encounetered the issue like `Cannot upload artifacts. You may have exceeded your spending limit for storing artifacts with Actions or Packages.` [2], but there still no expilict description in github official doc to show the real limited quota [3].
[1] https://github.com/actions/upload-artifact#retention-period
[2] https://github.com/actions/upload-artifact/issues/307
[3] https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts#about-workflow-artifacts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #42028:
URL: https://github.com/apache/spark/pull/42028#issuecomment-1637257452
cc @HyukjinKwon @dongjoon-hyun @Yikun @LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264789949
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
Review Comment:
ok, let me have a try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264795891
##########
.github/workflows/build_and_test.yml:
##########
@@ -686,6 +686,14 @@ jobs:
fi
cd docs
bundle exec jekyll build
+ cd ..
+ - name: Tar documentation
+ run: tar cjf site.tar.bz2 docs/_site
+ - name: Upload documentation
+ uses: actions/upload-artifact@v3
+ with:
+ name: site
Review Comment:
added this config, thanks for pointing it out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264821069
##########
dev/merge_spark_pr.py:
##########
@@ -44,7 +44,7 @@
# Location of your Spark git development area
SPARK_HOME = os.environ.get("SPARK_HOME", os.getcwd())
-# Remote name which points to the Gihub site
+# Remote name which points to the Github site
Review Comment:
@LuciferYang ok, let me send a separate minor PR for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zhengruifeng commented on a diff in pull request #42028: [SPARK-44451][BUILD] Make built document downloadable
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #42028:
URL: https://github.com/apache/spark/pull/42028#discussion_r1264835476
##########
dev/merge_spark_pr.py:
##########
@@ -44,7 +44,7 @@
# Location of your Spark git development area
SPARK_HOME = os.environ.get("SPARK_HOME", os.getcwd())
-# Remote name which points to the Gihub site
+# Remote name which points to the Github site
Review Comment:
https://github.com/apache/spark/pull/42032
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org