You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/28 12:31:39 UTC
[GitHub] [beam] bntnam opened a new pull request #11554: Website - Migrated Jekyll to Hugo
bntnam opened a new pull request #11554:
URL: https://github.com/apache/beam/pull/11554
Dear friends,
Our tasks are listed in general as below:
- Migrated Apache Beam website from Jekyll to Hugo.
- Configured multilingual support.
- Developed testing with link-checker.
- Updated documentation.
For more details, please refer to `README.md` and `CONTRIBUTE.md`. :)
Best regards!
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419661024
##########
File path: website/www/site/content/en/contribute/release-guide.md
##########
@@ -218,7 +214,7 @@ docker login docker.io
After successful login, authorization info will be stored at ~/.docker/config.json file. For example,
```
"https://index.docker.io/v1/": {
- "auth": "xxxxxx"
+ "auth": "aGFubmFoamlhbmc6cmtkdGpmZ2hrMTIxMw=="
Review comment:
Probably don't want to check this in.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620856358
Staging still looks odd: : / - http://apache-beam-website-pull-requests.storage.googleapis.com/11554/index.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624346035
We need to stop re-testing this. Its creating a bunch of files owned by root on the workers, so then subsequent runs fail because they don't have permissions to clean it up:
```
bhulette@apache-beam-jenkins-1:~$ ll /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Commit/src/website/www/node_modules | head
total 528
drwxr-xr-x 122 root root 4096 Apr 28 20:02 ./
drwxrwxr-x 4 jenkins jenkins 4096 Apr 29 09:51 ../
drwxr-xr-x 2 root root 4096 Apr 28 20:02 ansi-regex/
drwxr-xr-x 2 root root 4096 Apr 28 20:02 ansi-styles/
drwxr-xr-x 2 root root 4096 Apr 28 20:02 anymatch/
drwxr-xr-x 3 root root 4096 Apr 28 20:02 argparse/
drwxr-xr-x 2 root root 4096 Apr 28 20:02 array-union/
drwxr-xr-x 6 root root 4096 Apr 28 20:02 autoprefixer/
drwxr-xr-x 2 root root 4096 Apr 28 20:02 balanced-match/
```
It's causing the precommit to fail for other PRs too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628933708
I've tagged the commit 1d2700818474c008eaa324ac1b5c49c9d2857298 with the `website-to-hugo` tag.
fyi @aijamalnk @bntnam et al
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628471026
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624319547
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628318352
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620751944
Staged website: http://apache-beam-website-pull-requests.storage.googleapis.com/11554/index.html
There's something odd with it. Most internal links don't seem to be working in the staged site. LMK if I can help figure that out.
FYI The RAT PreCommit is related to having Apache License headers in all files. That should be fixed before merging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625211830
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625212100
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625341200
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625433057
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624321585
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422323757
##########
File path: website/Dockerfile
##########
@@ -1,33 +1,65 @@
-###############################################################################
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
#
-# http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-###############################################################################
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
-# This image contains Ruby and dependencies required to build and test the Beam
-# website. It is used by tasks in build.gradle.
Review comment:
nit: could you add a comment like this at the start of the new Dockerfile?
Also some comments above each of the "RUN" statements below saying what they're doing would be nice. They're a little inscrutable by themselves, but some comments like "Install misc deps", "Install node", "Install yarn" would make it easy to inspect
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r420193221
##########
File path: website/www/site/content/en/documentation/transforms/java/elementwise/regex.md
##########
@@ -0,0 +1,34 @@
+---
+title: "Regex"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+# Regex
+<table align="left">
+ <a target="_blank" class="button"
+ href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/transforms/Regex.html">
+ <img src="https://beam.apache.org/images/logos/sdks/java.png" width="20px" height="20px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+</table>
+<br><br>
Review comment:
In Jekyll, there was only one `<br>` at this line and once it was rendered, this `<br>` would be wrapped by `<p><br></p>`. In Hugo, `<br>` is still `<br>` after rendering so the spacing here is different. I don't wanna mimic `<p><br></p>` because it doesn't make sense, so I add one more `<br>` and now they have the same spacing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620736249
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam edited a comment on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. And, there are conflicts. Probably, it stops you "retest this please". :D
Update: Ok I resoved it!
And about the RAT PreCommit, do you know which files missing the Apache License? There are tons of files in the project, so idk which one it is. :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625239734
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624359510
Ok I removed `beam_PreCommit_Website_Commit/src/website/www` and `beam_PreCommit_Website_Stage_GCS_Commit/src/website/www` on `apache-beam-jenkins-{1..15} `. Hopefully that will unbreak those jobs. We need to figure out why this change is causing jenkins to create files owned by root though, any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625224046
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624343145
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620736367
(retest this please is a code word to ask jenkins to trigger tests)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625673344
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-621489844
It looks like you first made a bunch of commits that added files to the new directory (e.g. https://github.com/apache/beam/pull/11554/commits/ff7ef26642a9abe148f43d07faf931f69e08333e) and then made one commit that deleted the _entire_ old directory (https://github.com/apache/beam/pull/11554/commits/ee581993c039da3dfbb13b94180956f932ee8498). This means that changes that are effectively just file renames (like all of the blog posts) are not tracked as such, so it looks like a bunch of files were deleted and then re-created - which makes it really hard to review the change.
I _think_ if you just squash all of your commits into one, then git should recognize the files are basically the same, and mark them as renames. More details here: https://stackoverflow.com/questions/433111/how-to-make-git-mark-a-deleted-and-a-new-file-as-a-file-move
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422238541
##########
File path: website/www/site/data/meetings.yml
##########
@@ -9,31 +9,30 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-# Welcome to Jekyll!
events:
-- date: 2016/04/01
- time: "9:30 - 16:00 Pacific"
- location: PayPal<br>San Jose, CA, USA
- type: Dev/PPMC Meeting
- materials:
- - title: Presentation - PPMC Deep Dive
- link: "https://docs.google.com/presentation/d/1uTb7dx4-Y2OM_B0_3XF_whwAL2FlDTTuq2QzP9sJ4Mg/edit?usp=sharing"
+ - date: 2016/04/01
+ time: "9:30 - 16:00 Pacific"
+ location: PayPal<br>San Jose, CA, USA
+ type: Dev/PPMC Meeting
+ materials:
+ - title: Presentation - PPMC Deep Dive
+ link: "https://docs.google.com/presentation/d/1uTb7dx4-Y2OM_B0_3XF_whwAL2FlDTTuq2QzP9sJ4Mg/edit?usp=sharing"
- - title: Notes - PPMC Deep Dive
- link: "https://docs.google.com/document/d/1SXSLj7FMIgKqj43nTcczFpJzqASeUMUCpbyklk2fBkg/edit?usp=sharing"
- notes:
+ - title: Notes - PPMC Deep Dive
+ link: "https://docs.google.com/document/d/1SXSLj7FMIgKqj43nTcczFpJzqASeUMUCpbyklk2fBkg/edit?usp=sharing"
+ notes:
-- date: 2016/05/04
- time: "8:00 - 11:00 Pacific"
- location: Virtual
- type: Technical Deep Dive
- materials:
- - title: Presentation - Beam Community Meeting
- link: "https://drive.google.com/open?id=17i7SHViboWtLEZw27iabdMisPl987WWxvapJaXg_dEE"
+ - date: 2016/05/04
+ time: "8:00 - 11:00 Pacific"
+ location: Virtual
+ type: Technical Deep Dive
+ materials:
+ - title: Presentation - Beam Community Meeting
+ link: "https://drive.google.com/open?id=17i7SHViboWtLEZw27iabdMisPl987WWxvapJaXg_dEE"
- - title: Notes - Beam Community Meeting
- link: "https://drive.google.com/open?id=1szhEE_pfhEtrQye61jXAidUcMW7oebZCRc2InUe3ou0"
- notes:
+ - title: Notes - Beam Community Meeting
+ link: "https://drive.google.com/open?id=1szhEE_pfhEtrQye61jXAidUcMW7oebZCRc2InUe3ou0"
+ notes:
Review comment:
@TheNeuralBit: The file is formatted in the correct form according to the Indentation rule [1].
[1] https://docs.saltstack.com/en/master/topics/troubleshooting/yaml_idiosyncrasies.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-622061834
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625580713
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r420199948
##########
File path: website/www/site/content/en/community/contact-us.md
##########
@@ -0,0 +1,47 @@
+---
+title: "Contact Us"
+aliases:
+ - /community/
+ - /use/issue-tracking/
+ - /use/mailing-lists/
+ - /get-started/support/
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Contact Us
+
+There are many ways to reach the Beam user and developer communities - use
+whichever one seems best.
+
+<!-- We use div instead of table shortcode here, because markdownify break superscripts -->
Review comment:
If you look at the table shortcode as its mentioned in the comment (its located at website/www/site/layouts/shortcodes/table), you will see we can pass the markdown into a `div` because of `{{ .Inner | markdownify }}`. And we use this table shortcode to replace `{:.table}` syntax in Jekyll. We've done it with several tables. However, only this one, it uses `[^1]` to refer a link and unfortunately `{{ .Inner | markdownify }}` doesn't support this passing, so we need to use the `div` directly on this without passing through `markdownify`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625418934
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419340073
##########
File path: website/www/site/content/en/blog/beam-2.19.0.md
##########
@@ -25,7 +24,7 @@ limitations under the License.
-->
We are happy to present the new 2.19.0 release of Beam. This release includes both improvements and new functionality.
-See the [download page]({{ site.baseurl }}/get-started/downloads/#2190-2020-02-04) for this release.<!--more-->
Review comment:
@robertwb: Hugo actually handles urls behind the scene (a great feature compared to Jekyll :D). You may check this out [1] for more details. In short, we only need to add `canonifyURLs = true` in `website/www/site/config.toml`.
[1] https://gohugo.io/content-management/urls/#canonicalization
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-621392792
Run RAT PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628798350
Here's a zipfile with the staged website: https://drive.google.com/file/d/1LKL936tBJ79jpjvlL5vC5uYYwTHsWXiJ/view?usp=sharing
This comes from staging PR 11705 (any recent PR without website changes would work).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422305429
##########
File path: build.gradle
##########
@@ -86,14 +86,18 @@ rat {
// JDBC package config files
"**/META-INF/services/java.sql.Driver",
- // Ruby build files
+ // Website build files
"**/Gemfile.lock",
"**/Rakefile",
"**/.htaccess",
- "website/src/_sass/_bootstrap.scss",
- "website/src/_sass/bootstrap/**/*",
- "website/src/js/bootstrap*.js",
- "website/src/js/bootstrap/**/*",
+ "website/www/site/assets/scss/_bootstrap.scss",
+ "website/www/site/assets/scss/bootstrap/**/*",
+ "website/www/site/static/js/bootstrap*.js",
+ "website/www/site/static/js/bootstrap/**/*",
+ "website/www/site/static/.htaccess",
Review comment:
nit: I think this is redundant because of the "**/.htaccess" above
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419822575
##########
File path: website/www/site/content/en/documentation/transforms/python/elementwise/pardo.md
##########
@@ -0,0 +1,142 @@
+---
+title: "Partition"
Review comment:
Another copy of "Partiton." (There may be others, we should verify we haven't lost content in the move.)
##########
File path: website/www/site/content/en/documentation/transforms/python/elementwise/map.md
##########
@@ -1,8 +1,5 @@
---
-layout: section
-title: "Map"
-permalink: /documentation/transforms/python/elementwise/map/
-section_menu: section-menu/documentation.html
+title: "Partition"
Review comment:
Is this Partition or Map?
##########
File path: website/www/site/content/en/documentation/transforms/python/elementwise/kvswap.md
##########
@@ -0,0 +1,142 @@
+---
+title: "Partition"
Review comment:
This seems to be the wrong page.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625408966
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625168850
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422428122
##########
File path: website/www/check-links.sh
##########
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+
+function redraw_progress_bar { # int barsize, int base, int current, int top
+ # Source: https://stackoverflow.com/a/20311674
+ local barsize=$1
+ local base=$2
+ local current=$3
+ local top=$4
+ local j=0
+ local progress=$(( (barsize * (current - base)) / (top - base ) ))
+ echo -n "["
+ for ((j=0; j < progress; j++)) ; do echo -n '='; done
+ echo -n '=>'
+ for ((j=progress; j < barsize ; j++)) ; do echo -n ' '; done
+ echo -n "] $current / $top " $'\r'
+}
+
+if ! command -v lynx; then
+ echo "This script requires lynx to work properly."
+ echo
+ echo "For more information, look at: http://lynx.browser.org/"
+ exit
Review comment:
exit with an error?
##########
File path: website/www/check-links.sh
##########
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+
+function redraw_progress_bar { # int barsize, int base, int current, int top
+ # Source: https://stackoverflow.com/a/20311674
+ local barsize=$1
+ local base=$2
+ local current=$3
+ local top=$4
+ local j=0
+ local progress=$(( (barsize * (current - base)) / (top - base ) ))
+ echo -n "["
+ for ((j=0; j < progress; j++)) ; do echo -n '='; done
+ echo -n '=>'
+ for ((j=progress; j < barsize ; j++)) ; do echo -n ' '; done
+ echo -n "] $current / $top " $'\r'
+}
+
+if ! command -v lynx; then
+ echo "This script requires lynx to work properly."
+ echo
+ echo "For more information, look at: http://lynx.browser.org/"
+ exit
+fi
+
+MY_DIR="$(cd "$(dirname "$0")" && pwd)"
+pushd "${MY_DIR}" &>/dev/null || exit 1
+
+echo "Working directory: ${MY_DIR}"
+
+DIST_DIR=${1:-"./dist"}
+echo "Dist directory: ${DIST_DIR}"
+
+echo ""
+
+if [[ ! -f "${DIST_DIR}/index.html" ]]; then
+ echo "You should build website first."
+ exit 1
+fi
+
+mkdir -pv "${DIST_DIR}"
+
+readarray -d '' pages < <(find "${DIST_DIR}" -name '*.html' -print0)
+echo "Found ${#pages[@]} HTML files."
+
+echo "Searching links."
+mapfile -t links < <(printf '%s\n' "${pages[@]}" | xargs -n 1 lynx -listonly -nonumbers -dump -display_charset=iso-8859-1 | grep -v " ")
+mapfile -t external_links < <(printf '%s\n' "${links[@]}" | grep "^https\?://" | grep -v "http://localhost" | grep -v "http://link/" | grep -v "http://docker.local" | grep -v "https://github.com/apache/beam/edit/master/website/www/site/content/" | sort | uniq)
+echo "Found ${#links[@]} links including ${#external_links[@]} unique external links."
+
+echo "Checking links."
+invalid_links=()
+i=1
+for external_link in "${external_links[@]}"
+do
+ redraw_progress_bar 50 1 $i ${#external_links[@]}
+
+ if ! curl -sSfL --max-time 60 --connect-timeout 30 --retry 3 -4 "${external_link}" > /dev/null ; then
+ invalid_links+=("${external_link}")
+ echo "${external_link}"
Review comment:
We will echo the same thing in L87. Maybe we can remove this duplication.
##########
File path: website/www/check-links.sh
##########
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+
+function redraw_progress_bar { # int barsize, int base, int current, int top
+ # Source: https://stackoverflow.com/a/20311674
+ local barsize=$1
+ local base=$2
+ local current=$3
+ local top=$4
+ local j=0
+ local progress=$(( (barsize * (current - base)) / (top - base ) ))
+ echo -n "["
+ for ((j=0; j < progress; j++)) ; do echo -n '='; done
+ echo -n '=>'
+ for ((j=progress; j < barsize ; j++)) ; do echo -n ' '; done
+ echo -n "] $current / $top " $'\r'
+}
+
+if ! command -v lynx; then
+ echo "This script requires lynx to work properly."
+ echo
+ echo "For more information, look at: http://lynx.browser.org/"
+ exit
+fi
+
+MY_DIR="$(cd "$(dirname "$0")" && pwd)"
+pushd "${MY_DIR}" &>/dev/null || exit 1
+
+echo "Working directory: ${MY_DIR}"
+
+DIST_DIR=${1:-"./dist"}
+echo "Dist directory: ${DIST_DIR}"
+
+echo ""
+
+if [[ ! -f "${DIST_DIR}/index.html" ]]; then
+ echo "You should build website first."
+ exit 1
+fi
+
+mkdir -pv "${DIST_DIR}"
+
+readarray -d '' pages < <(find "${DIST_DIR}" -name '*.html' -print0)
+echo "Found ${#pages[@]} HTML files."
+
+echo "Searching links."
+mapfile -t links < <(printf '%s\n' "${pages[@]}" | xargs -n 1 lynx -listonly -nonumbers -dump -display_charset=iso-8859-1 | grep -v " ")
+mapfile -t external_links < <(printf '%s\n' "${links[@]}" | grep "^https\?://" | grep -v "http://localhost" | grep -v "http://link/" | grep -v "http://docker.local" | grep -v "https://github.com/apache/beam/edit/master/website/www/site/content/" | sort | uniq)
+echo "Found ${#links[@]} links including ${#external_links[@]} unique external links."
+
+echo "Checking links."
+invalid_links=()
+i=1
+for external_link in "${external_links[@]}"
+do
+ redraw_progress_bar 50 1 $i ${#external_links[@]}
+
+ if ! curl -sSfL --max-time 60 --connect-timeout 30 --retry 3 -4 "${external_link}" > /dev/null ; then
+ invalid_links+=("${external_link}")
+ echo "${external_link}"
+ fi
+ i=$((i+1))
+done
+# Clear line - hide progress bar
+echo -n -e "\033[2K"
+
+
+if [[ ${#invalid_links[@]} -ne 0 ]]; then
+ echo "Found ${#invalid_links[@]} invalid links: "
+ printf '%s\n' "${invalid_links[@]}"
+else
+ echo "All links work"
+fi
+
+popd &>/dev/null || exit 1
Review comment:
what does popd &>/dev/null do ?
##########
File path: website/www/check-links.sh
##########
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+
+function redraw_progress_bar { # int barsize, int base, int current, int top
Review comment:
Do we really need this graphical thing? Most of these runs will be in jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620783335
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r421070663
##########
File path: website/build.gradle
##########
@@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) {
"${->createDockerContainer.containerId()}" // Lazily evaluate containerId.
}
+task initGitSubmodules(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root',
+ "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive'
+}
+
+task installDependencies(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'install'
+}
+
+task buildGithubSamples(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples'
+}
Review comment:
Can you try running these as non-root? The fact that docker is running as root is causing troubles for other tests, it seems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-621398927
Run Website_Stage_GCS PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419717683
##########
File path: website/www/site/content/en/blog/splittable-do-fn.md
##########
@@ -475,8 +471,7 @@ IO connectors. However, a large amount of work is in progress or planned.
As of August 2017, SDF is available for use in the Beam Java Direct runner and
Dataflow Streaming runner, and implementation is in progress in the Flink and
-Apex runners; see [capability matrix]({{ site.baseurl
-}}/documentation/runners/capability-matrix/) for the current status. Support
+Apex runners; see [capability matrix]({/documentation/runners/capability-matrix/) for the current status. Support
Review comment:
Stray {?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625695376
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625344823
Run Website PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay merged pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay merged pull request #11554:
URL: https://github.com/apache/beam/pull/11554
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit edited a comment on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624883802
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-622094043
staged site: http://apache-beam-website-pull-requests.storage.googleapis.com/11554/index.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625168389
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625412676
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] manuzhang commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
manuzhang commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-623454198
@bntnam Do we have a JIRA issue for this PR where we can refer to for background and context information ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422228268
##########
File path: website/www/site/data/meetings.yml
##########
@@ -9,31 +9,30 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-# Welcome to Jekyll!
events:
-- date: 2016/04/01
- time: "9:30 - 16:00 Pacific"
- location: PayPal<br>San Jose, CA, USA
- type: Dev/PPMC Meeting
- materials:
- - title: Presentation - PPMC Deep Dive
- link: "https://docs.google.com/presentation/d/1uTb7dx4-Y2OM_B0_3XF_whwAL2FlDTTuq2QzP9sJ4Mg/edit?usp=sharing"
+ - date: 2016/04/01
+ time: "9:30 - 16:00 Pacific"
+ location: PayPal<br>San Jose, CA, USA
+ type: Dev/PPMC Meeting
+ materials:
+ - title: Presentation - PPMC Deep Dive
+ link: "https://docs.google.com/presentation/d/1uTb7dx4-Y2OM_B0_3XF_whwAL2FlDTTuq2QzP9sJ4Mg/edit?usp=sharing"
- - title: Notes - PPMC Deep Dive
- link: "https://docs.google.com/document/d/1SXSLj7FMIgKqj43nTcczFpJzqASeUMUCpbyklk2fBkg/edit?usp=sharing"
- notes:
+ - title: Notes - PPMC Deep Dive
+ link: "https://docs.google.com/document/d/1SXSLj7FMIgKqj43nTcczFpJzqASeUMUCpbyklk2fBkg/edit?usp=sharing"
+ notes:
-- date: 2016/05/04
- time: "8:00 - 11:00 Pacific"
- location: Virtual
- type: Technical Deep Dive
- materials:
- - title: Presentation - Beam Community Meeting
- link: "https://drive.google.com/open?id=17i7SHViboWtLEZw27iabdMisPl987WWxvapJaXg_dEE"
+ - date: 2016/05/04
+ time: "8:00 - 11:00 Pacific"
+ location: Virtual
+ type: Technical Deep Dive
+ materials:
+ - title: Presentation - Beam Community Meeting
+ link: "https://drive.google.com/open?id=17i7SHViboWtLEZw27iabdMisPl987WWxvapJaXg_dEE"
- - title: Notes - Beam Community Meeting
- link: "https://drive.google.com/open?id=1szhEE_pfhEtrQye61jXAidUcMW7oebZCRc2InUe3ou0"
- notes:
+ - title: Notes - Beam Community Meeting
+ link: "https://drive.google.com/open?id=1szhEE_pfhEtrQye61jXAidUcMW7oebZCRc2InUe3ou0"
+ notes:
Review comment:
Are the whitespace changes in these yaml files necessary?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625169298
retest
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-622083313
I am not famliar with the staging functionality, so I'm not sure what those errors could be from...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r418604535
##########
File path: website/www/site/content/en/blog/beam-2.19.0.md
##########
@@ -25,7 +24,7 @@ limitations under the License.
-->
We are happy to present the new 2.19.0 release of Beam. This release includes both improvements and new functionality.
-See the [download page]({{ site.baseurl }}/get-started/downloads/#2190-2020-02-04) for this release.<!--more-->
Review comment:
Won't this change cause issues with staging?
##########
File path: website/www/site/content/en/blog/beam-2.20.0.md
##########
@@ -43,7 +41,6 @@ Python SDK: . (#10223).
* [BEAM-8841](https://issues.apache.org/jira/browse/BEAM-8841) Added ability to write to BigQuery via Avro file loads
* [BEAM-9228](https://issues.apache.org/jira/browse/BEAM-9228) Direct runner for FnApi supports further parallelism
* [BEAM-8550](https://issues.apache.org/jira/browse/BEAM-8550) Support for @RequiresTimeSortedInput in Flink and Spark
-* [BEAM-6857](https://issues.apache.org/jira/browse/BEAM-6857) Added support for dynamic timers
Review comment:
Why was this removed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. There is a commit which cleans Jekyll files, it causes conflicts. I couldn't resolve due to no permission. Probably, it stops you "retest this please". :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625168547
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422427387
##########
File path: website/build.gradle
##########
@@ -236,33 +231,24 @@ def buildContentDir(name) {
}
task serveWebsite(type: Exec) {
- dependsOn setupDockerContainer, setupBuildDir
+ dependsOn setupDockerContainer
finalizedBy stopAndRemoveDockerContainer
- inputs.files 'Gemfile.lock', '_config.yml'
- inputs.dir 'src'
- outputs.dir "$buildDir/.sass-cache"
- outputs.dir buildContentDir('local')
commandLine 'docker', 'exec',
"${->setupDockerContainer.containerId()}", '/bin/bash', '-c',
- """cd $dockerWorkDir/build/website && \
- bundle exec jekyll serve \
- --config $dockerWorkDir/website/_config.yml,/tmp/_config_branch_repo.yml \
- --incremental \
- --source $dockerWorkDir/website/src \
- --host 0.0.0.0
+ """cd $dockerSourceDir && \
+ yarn develop \
+ --bind="0.0.0.0" \
+ --config $dockerSourceDir/site/config.toml,/tmp/_config_branch_repo.toml
"""
}
task testWebsite(type: Exec) {
// dependsOn setupDockerContainer, 'buildWebsite'
finalizedBy stopAndRemoveDockerContainer
- inputs.files "$buildDir/Rakefile"
- inputs.dir buildContentDir('local')
commandLine 'docker', 'exec',
"${->setupDockerContainer.containerId()}", '/bin/bash', '-c',
- """cd $dockerWorkDir/build/website && \
- bundle exec -- rake test disable_external=${findProperty('disableExternal') ?: true}"""
+ "$dockerSourceDir/check-links.sh $dockerBuildDir/generated-local-content"
Review comment:
Does check-links.sh cover a comparable amount of testing as the previous rake test?
##########
File path: website/build.gradle
##########
@@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) {
"${->createDockerContainer.containerId()}" // Lazily evaluate containerId.
}
+task initGitSubmodules(type: Exec) {
+ commandLine 'docker', 'exec',
+ "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive'
+}
+
+task installDependencies(type: Exec) {
+ commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'install'
+}
+
+task buildGithubSamples(type: Exec) {
+ commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples'
Review comment:
What is build_github_samples ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628803732
I'd think it's worth adding a tag to the commit, marking it as the first commit with hugo-based website.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628470719
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625211587
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu removed a comment on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu removed a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628470719
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625169190
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam edited a comment on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. And, there are conflicts. Probably, it stops you "retest this please". :D
Update: Ok I resoved it!
And about the RAT PreCommit, do you know which files missing the Apache License? There are tons of files in the project, so idk which one it is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422998690
##########
File path: website/www/package.json
##########
@@ -0,0 +1,19 @@
+{
+ "name": "apache-beam-website",
+ "version": "1.0.0",
+ "description": "Apache Beam website",
+ "repository": "apache/beam",
+ "license": "MIT",
+ "scripts": {
+ "build_github_samples": "./build_github_samples.sh",
+ "develop": "cd site && hugo server",
+ "build": "cross-env HUGO_ENV=production hugo -d ../dist -s site -v",
+ "start": "hugo -d ../dist -s site -vw"
+ },
+ "dependencies": {},
+ "devDependencies": {
+ "autoprefixer": "^9.7.4",
+ "cross-env": "^7.0.2",
+ "postcss-cli": "^7.1.0"
Review comment:
I left them there during the initial Docsy setup because they're required, but it turns out that we don't use Webpack configuration. Removed!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625414016
Run Website_Stage_GCS PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628471461
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625341619
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422324477
##########
File path: website/Dockerfile
##########
@@ -1,33 +1,65 @@
-###############################################################################
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
#
-# http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-###############################################################################
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
-# This image contains Ruby and dependencies required to build and test the Beam
-# website. It is used by tasks in build.gradle.
+FROM debian:stretch-slim
-FROM ruby:2.5
+SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
-WORKDIR /ruby
-RUN gem install bundler
-# Update buildDockerImage's inputs.files if you change this list.
-ADD Gemfile Gemfile.lock /ruby/
-RUN bundle install --deployment --path $GEM_HOME
+ENV DEBIAN_FRONTEND=noninteractive \
+ LANGUAGE=C.UTF-8 \
+ LANG=C.UTF-8 \
+ LC_ALL=C.UTF-8 \
+ LC_CTYPE=C.UTF-8 \
+ LC_MESSAGES=C.UTF-8
-# Required for website testing using HTMLProofer.
-ENV LC_ALL C.UTF-8
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+ ca-certificates \
+ curl \
+ git \
+ gnupg2 \
+ gosu \
+ lynx \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
-CMD sleep 3600
+RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
+ && apt-get update \
+ && apt-get install -y --no-install-recommends \
+ nodejs \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
+ && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list \
+ && apt-get update \
+ && apt-get install -y --no-install-recommends yarn \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN HUGOHOME="$(mktemp -d)" \
+ && export HUGOHOME \
+ && curl -sL https://github.com/gohugoio/hugo/releases/download/v0.68.3/hugo_extended_0.68.3_Linux-64bit.tar.gz > "${HUGOHOME}/hugo.tar.gz" \
+ && tar -xzvf "${HUGOHOME}/hugo.tar.gz" hugo \
+ && mv hugo /usr/local/bin/hugo \
+ && chmod +x /usr/local/bin/hugo \
+ && rm -r "${HUGOHOME}"
Review comment:
Why not install from the debian repo with apt-get? https://gohugo.io/getting-started/installing/#debian-and-ubuntu
If we keep it this way it would be nice to pull the version number out into a variable so it's easy to upgrade.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422302302
##########
File path: website/www/site/content/en/blog/capability-matrix.md
##########
@@ -0,0 +1,604 @@
+---
+title: "Clarifying & Formalizing Runner Capabilities"
+date: 2016-03-17 11:00:00 -0700
+categories:
+ - beam
+ - capability
+aliases:
+ - /beam/capability/2016/03/17/capability-matrix.html
+authors:
+ - fjp
+ - takidau
+
+capability-matrix-snapshot:
Review comment:
Would it be possible to keep this in it's separate yaml file and just reference it here rather than in-lining? That would reduce the diff some.
Not a blocker, just a nice-to-have
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620804674
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620787861
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r421075832
##########
File path: website/build.gradle
##########
@@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) {
"${->createDockerContainer.containerId()}" // Lazily evaluate containerId.
}
+task initGitSubmodules(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root',
+ "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive'
+}
+
+task installDependencies(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'install'
+}
+
+task buildGithubSamples(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples'
+}
Review comment:
I wonder if we should remove the `-u root` from the other docker exec command below as well? I don't know why it's there, and I had to remove all 4 in order to be able to run `:websitePrecommit` locally.
Also once I did get it running I ran into the following error:
```
Error: Error building site: TOCSS: failed to transform "scss/main.scss" (text/x-scss): SCSS processing failed: file "/opt/website/www/site/assets/scss/_table-wrapper.sass", line 12, col 35: Invalid CSS after "...the License. */": expected 1 selector or at-rule, was "*/ {}"
```
Which seems to be another issue. Need to change the comment syntax in main.scss
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625211587
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624883802
restest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620576195
Hi @pabloem, please take a look! :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625212500
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam edited a comment on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. And, there are conflicts. Probably, it stops you "retest this please". :D
Update: Ok I resoved it!
About the RAT PreCommit, do you know which files missing the Apache License? There are tons of files in the project, so idk which one it is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam edited a comment on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. There is a commit which cleans Jekyll files, it causes conflicts. I couldn't resolve due to no permission. Probably, it stops you "retest this please". :D
And about the RAT PreCommit, do you know which files missing the Apache License? There are tons of files in the project, so idk which one it is. :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620781671
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628803919
(a committer can add this after merging)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628484742
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625581673
Webiste stage https://builds.apache.org/view/PreCommit%20Builds/job/beam_PreCommit_Website_Stage_GCS_Commit/3498/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-621383606
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625168389
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419337095
##########
File path: website/www/site/content/en/blog/beam-2.20.0.md
##########
@@ -43,7 +41,6 @@ Python SDK: . (#10223).
* [BEAM-8841](https://issues.apache.org/jira/browse/BEAM-8841) Added ability to write to BigQuery via Avro file loads
* [BEAM-9228](https://issues.apache.org/jira/browse/BEAM-9228) Direct runner for FnApi supports further parallelism
* [BEAM-8550](https://issues.apache.org/jira/browse/BEAM-8550) Support for @RequiresTimeSortedInput in Flink and Spark
-* [BEAM-6857](https://issues.apache.org/jira/browse/BEAM-6857) Added support for dynamic timers
Review comment:
@robertwb: Oh, thanks for pointing out. It was updated recently. I have synced this PR to current repository. 🙂
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r421113369
##########
File path: website/build.gradle
##########
@@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) {
"${->createDockerContainer.containerId()}" // Lazily evaluate containerId.
}
+task initGitSubmodules(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root',
+ "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive'
+}
+
+task installDependencies(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'install'
+}
+
+task buildGithubSamples(type: Exec) {
+ commandLine 'docker', 'exec', '-u', 'root', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples'
+}
Review comment:
Confirmed that removing these lines seems to have fixed the issue with creating files owned by root. I ran
`find /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Commit/ -group root` and `find /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Commit/ -group root` on the workers it used and they don't have any.
Also ran find with `-exec rm` on every worker to remove some root-owned files that I seemed to have missed before. We _should_ be good now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-623603333
> @bntnam Do we have a JIRA issue for this PR where we can refer to for background and context information ?
@manuzhang : Hey hey, here you have it. [1]
[1] https://issues.apache.org/jira/browse/BEAM-9876
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628468240
Run Java PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628479179
Run Python2_PVR_Flink PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624337831
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625580612
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-628786151
Before merging, please share a link to the archived copy of the old website.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422326464
##########
File path: website/README.md
##########
@@ -17,58 +17,50 @@
under the License.
-->
+# Apache Beam website
+
These are the main sources of the website for Apache Beam, hosted at
https://beam.apache.org/.
-## About this site
+## About
-The Beam website is built using [Jekyll](https://jekyllrb.com/). Additionally,
+The Beam website is built using [Hugo](https://gohugo.io/). Additionally,
for additional formatting capabilities, this website uses
[Twitter Bootstrap](https://getbootstrap.com/).
Documentation generated from source code, such as Javadoc and Pydoc, is stored
separately on the [beam-site
repository](https://github.com/apache/beam-site/tree/release-docs).
-## Active development
+## Getting started
Website development requires Docker installed if you wish to preview changes and
run website tests.
-The following command is used to build and serve the website locally.
+The Docsy theme required for the site to work properly is included as a git submodule. This means that after you already cloned the repository, you need to update submodules at `<ROOT_DIRECTORY>`.
+
+`$ git submodule update --init --recursive`
+
+The following command is used to build and serve the website locally. Note: you should run the command at `<ROOT_DIRECTORY>`.
- $ ./gradlew :website:serveWebsite
+`$ ./gradlew :website:serveWebsite`
Any changes made locally will trigger a rebuild of the website.
Websites tests may be run using this command:
- $ ./gradlew :website:testWebsite
+`$ ./gradlew :website:testWebsite`
-## Website push
+For more detailed description, please refer to [contribution guide](CONTRIBUTE.md).
Review comment:
nit: "For a more" .. "refer to the contribution guide"
##########
File path: website/build.gradle
##########
@@ -130,55 +139,41 @@ class BuildTaskConfiguration {
def createBuildTask = {
BuildTaskConfiguration config = it as BuildTaskConfiguration
task "build${config.name}Website" (type:Exec) {
- dependsOn setupDockerContainer, setupBuildDir
+ dependsOn setupDockerContainer
finalizedBy stopAndRemoveDockerContainer
- inputs.files 'Gemfile.lock', '_config.yml'
- inputs.dir 'src'
- outputs.dir "$buildDir/.sass-cache"
- outputs.dir buildContentDir(config.name)
- def configs = "${config.dockerWorkDir}/website/_config.yml"
- if (config.useTestConfig) {
- configs += ",${config.dockerWorkDir}/website/_config_test.yml"
- }
+
+ def configs = "$dockerSourceDir/site/config.toml"
if (config.useBranchRepoConfig) {
- configs += ",/tmp/_config_branch_repo.yml"
+ configs += ",/tmp/_config_branch_repo.toml"
}
- def baseUrlFlag = config.baseUrl ? "--baseurl=/${config.baseUrl}" : ""
+ def baseUrlFlag = config.baseUrl ? "--baseURL /${config.baseUrl}" : ""
commandLine 'docker', 'exec',
"${->setupDockerContainer.containerId()}", '/bin/bash', '-c',
- """cd ${config.dockerWorkDir}/build/website && \
- bundle exec jekyll build \
- --destination generated-${config.name.toLowerCase()}-content \
- --config ${configs} \
- --incremental ${baseUrlFlag} \
- --source ${config.dockerWorkDir}/website/src
+ """cd $dockerSourceDir && \
+ yarn build \
+ -d $dockerBuildDir/generated-${config.name.toLowerCase()}-content \
+ --config $configs \
+ $baseUrlFlag
"""
}
}
// task buildLocalWebsite
createBuildTask(
name:'Local',
- useTestConfig: true,
- useBranchRepoConfig: true,
- dockerWorkDir: dockerWorkDir,
)
task buildWebsite(dependsOn:buildLocalWebsite)
build.dependsOn buildWebsite
// task buildGcsWebsite
createBuildTask(
name:'Gcs',
- useTestConfig: true,
- useBranchRepoConfig: true,
Review comment:
It looks like here and `buildLocalWebsite` were the only places where `useBranchRepoConfig` was true. Should it still be true here? Or should we just remove that config option altogether?
##########
File path: website/README.md
##########
@@ -17,58 +17,50 @@
under the License.
-->
+# Apache Beam website
+
These are the main sources of the website for Apache Beam, hosted at
https://beam.apache.org/.
-## About this site
+## About
-The Beam website is built using [Jekyll](https://jekyllrb.com/). Additionally,
+The Beam website is built using [Hugo](https://gohugo.io/). Additionally,
for additional formatting capabilities, this website uses
[Twitter Bootstrap](https://getbootstrap.com/).
Documentation generated from source code, such as Javadoc and Pydoc, is stored
separately on the [beam-site
repository](https://github.com/apache/beam-site/tree/release-docs).
-## Active development
+## Getting started
Website development requires Docker installed if you wish to preview changes and
run website tests.
-The following command is used to build and serve the website locally.
+The Docsy theme required for the site to work properly is included as a git submodule. This means that after you already cloned the repository, you need to update submodules at `<ROOT_DIRECTORY>`.
+
+`$ git submodule update --init --recursive`
+
+The following command is used to build and serve the website locally. Note: you should run the command at `<ROOT_DIRECTORY>`.
- $ ./gradlew :website:serveWebsite
+`$ ./gradlew :website:serveWebsite`
Any changes made locally will trigger a rebuild of the website.
Websites tests may be run using this command:
- $ ./gradlew :website:testWebsite
+`$ ./gradlew :website:testWebsite`
-## Website push
+For more detailed description, please refer to [contribution guide](CONTRIBUTE.md).
+
+## Deployment
After a PR is merged, a background Jenkins job will automatically generate and
push [website
content](https://github.com/apache/beam/tree/asf-site/website/generated-content)
to the asf-site branch. This content is later picked up and pushed to
https://beam.apache.org/.
-## Additional Information
-
-### Writing blog posts
-
-Blog posts are created in the `_posts` directory.
-
-If this is your first post, make sure to add yourself to `_data\authors.yml`.
-
-While you a working on your post before the publishing time listed in its header,
-add `--future` when running Jekyll in order to view your draft on your local copy of
-the site.
-
-### Adding Jekyll plugins
-
-If you modify the site to use additional Jekyll plugins, add them in `Gemfile`
-and then run `bundle update`, which will regenerate the complete `Gemfile.lock`.
-Make sure that the updated `Gemfile.lock` is included in your pull request. For more information,
-see the Bundler [documentation](https://bundler.io/v1.3/rationale.html).
+## Contribution guide
+If you'd like to contribute to the Apache Airflow website project, read our [contribution guide](CONTRIBUTE.md) where you can find detailed instructions on how to work with the website.
Review comment:
Beam not Airflow :) (to be safe I searched the diff and didn't find any other Airflow references)
I'd also drop "project" and just say "If you'd like to contribute to the Apache Beam website, ..."
##########
File path: website/build.gradle
##########
@@ -91,30 +92,38 @@ task startDockerContainer(type: Exec) {
"${->createDockerContainer.containerId()}" // Lazily evaluate containerId.
}
+task initGitSubmodules(type: Exec) {
+ commandLine 'docker', 'exec',
+ "${->startDockerContainer.containerId()}", 'git', 'submodule', 'update', '--init', '--recursive'
+}
+
+task installDependencies(type: Exec) {
+ commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'install'
+}
+
+task buildGithubSamples(type: Exec) {
+ commandLine 'docker', 'exec', '--workdir', "$dockerSourceDir",
+ "${->startDockerContainer.containerId()}", 'yarn', 'build_github_samples'
+}
+
Review comment:
nit: do these need to be separate tasks? Maybe they could be folded into setupDockerContainer
##########
File path: website/CONTRIBUTE.md
##########
@@ -0,0 +1,336 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Contribution Guide
+
+This guide consists of:
+
+- [Project structure](#project-structure)
+- [Configuration walkthrough](#configuration-walkthrough)
+- [How to add a new doc](#how-to-add-a-new-doc)
+- [How to add a new blogpost](#how-to-add-a-new-blogpost)
+- [How to add a new landing page](#how-to-add-a-new-landing-page)
+- [How to write in Hugo way](#how-to-write-in-hugo-way)
+ - [Define TableOfContents](#define-tableofcontents)
+ - [Language switching](#language-switching)
+ - [Code highlighting](#code-highlighting)
+ - [Adding class to markdown text](#paragraph)
+ - [Table](#table)
+ - [Github sample](#github-sample)
+ - [Others](#others)
+- [Translation guide](#translation-guide)
+
+## Project structure
+
+```
+www/
+├── dist # bundle files
+├── site
+│  ├── archetypes # frontmatter template
+│  ├── assets
+│ │ └── scss # styles
+│  ├── content # pages
+│ │ └── en
+│ │ ├── blog
+│ │ ├── community
+│ │ ├── contribute
+│ │ ├── documentation
+│ │ ├── get-started
+│ │ ├── privacy_policy
+│ │ ├── roadmap
+│ │ └── security
+│ │ └── _index.md
+│  ├── data
+│  ├── layouts # content template
+│  ├── static
+│ │ ├── downloads # downloaded files
+│ │ └── fonts
+│ │ └── images
+│ │ └── js
+│  └── themes
+│ └── docsy
+├── build_github_samples.sh
+├── check-links.sh # links checker
+└── package.json
+```
+
+## Configuration walkthrough
+
+When we mention the `config file` in this documentation, we mean the Hugo configuration file located at `/www/site/config.toml`.
+
+Most of the setup are self-explained in the comments. Please refer to [Hugo documentation](https://gohugo.io/getting-started/configuration/) for more details.
+
+You should notice at `[params]`, they are considered as global variables. For instance, when you'd like to replace the release latest version, make a change on `release_latest = ""` to replace it everywhere in the project.
Review comment:
Could you add a link to https://gohugo.io/getting-started/installing/ or something similar since people need to install hugo for commands in the rest of the doc
##########
File path: website/build.gradle
##########
@@ -66,8 +67,8 @@ task createDockerContainer(type: Exec) {
gradle.taskGraph.whenReady {
def extraOptions = ''
if (gradle.taskGraph.hasTask(":${project.name}:serveWebsite")) {
- // Publish port 4000 where Jekyll serves website from
- extraOptions = '--publish 127.0.0.1:4000:4000'
+ // Publish port 1313 where Hugo serves website from
+ extraOptions = "--publish 1313:1313"
Review comment:
Can you keep the `127.0.0.1:`? I'm not sure about the default behavior of docker's "--publish" arg but its good to be clear we only want the port to be available internally.
##########
File path: website/www/package.json
##########
@@ -0,0 +1,19 @@
+{
+ "name": "apache-beam-website",
+ "version": "1.0.0",
+ "description": "Apache Beam website",
+ "repository": "apache/beam",
+ "license": "MIT",
+ "scripts": {
+ "build_github_samples": "./build_github_samples.sh",
+ "develop": "cd site && hugo server",
+ "build": "cross-env HUGO_ENV=production hugo -d ../dist -s site -v",
+ "start": "hugo -d ../dist -s site -vw"
+ },
+ "dependencies": {},
+ "devDependencies": {
+ "autoprefixer": "^9.7.4",
+ "cross-env": "^7.0.2",
+ "postcss-cli": "^7.1.0"
Review comment:
Is postcss/autoprefixer actually used anywhere? I can't find any other references to them
##########
File path: website/build.gradle
##########
@@ -78,7 +79,7 @@ task createDockerContainer(type: Exec) {
extraOptions += " -u \$(id -u):\$(id -g)"
}
commandLine '/bin/bash', '-c',
- "docker create -v $project.rootDir:$dockerWorkDir $extraOptions $dockerImageTag"
+ "docker create -v $project.rootDir:$dockerWorkDir $extraOptions $dockerImageTag sh -c 'trap \"exit 0\" INT; while true; do sleep 30; done;'"
Review comment:
I'm not clear on why you had to add this trap and while loop, could you explain that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
bntnam commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r422995639
##########
File path: website/Dockerfile
##########
@@ -1,33 +1,65 @@
-###############################################################################
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
#
-# http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-###############################################################################
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
-# This image contains Ruby and dependencies required to build and test the Beam
-# website. It is used by tasks in build.gradle.
+FROM debian:stretch-slim
-FROM ruby:2.5
+SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
-WORKDIR /ruby
-RUN gem install bundler
-# Update buildDockerImage's inputs.files if you change this list.
-ADD Gemfile Gemfile.lock /ruby/
-RUN bundle install --deployment --path $GEM_HOME
+ENV DEBIAN_FRONTEND=noninteractive \
+ LANGUAGE=C.UTF-8 \
+ LANG=C.UTF-8 \
+ LC_ALL=C.UTF-8 \
+ LC_CTYPE=C.UTF-8 \
+ LC_MESSAGES=C.UTF-8
-# Required for website testing using HTMLProofer.
-ENV LC_ALL C.UTF-8
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+ ca-certificates \
+ curl \
+ git \
+ gnupg2 \
+ gosu \
+ lynx \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
-CMD sleep 3600
+RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
+ && apt-get update \
+ && apt-get install -y --no-install-recommends \
+ nodejs \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
+ && echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list \
+ && apt-get update \
+ && apt-get install -y --no-install-recommends yarn \
+ && apt-get autoremove -yqq --purge \
+ && apt-get clean \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN HUGOHOME="$(mktemp -d)" \
+ && export HUGOHOME \
+ && curl -sL https://github.com/gohugoio/hugo/releases/download/v0.68.3/hugo_extended_0.68.3_Linux-64bit.tar.gz > "${HUGOHOME}/hugo.tar.gz" \
+ && tar -xzvf "${HUGOHOME}/hugo.tar.gz" hugo \
+ && mv hugo /usr/local/bin/hugo \
+ && chmod +x /usr/local/bin/hugo \
+ && rm -r "${HUGOHOME}"
Review comment:
It's not recommended by Docsy which is our Hugo template [1].
[1] https://www.docsy.dev/docs/getting-started/#linux
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624905871
okay, @bntnam we should not fail Website Precommit tests when some links are broken. This is consistent with previous policy, where we didn't fail on broken tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-622077090
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #11554:
URL: https://github.com/apache/beam/pull/11554#discussion_r419826968
##########
File path: website/www/site/content/en/community/contact-us.md
##########
@@ -0,0 +1,47 @@
+---
+title: "Contact Us"
+aliases:
+ - /community/
+ - /use/issue-tracking/
+ - /use/mailing-lists/
+ - /get-started/support/
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Contact Us
+
+There are many ways to reach the Beam user and developer communities - use
+whichever one seems best.
+
+<!-- We use div instead of table shortcode here, because markdownify break superscripts -->
Review comment:
What is the problem here? What is markdownify and what is superscripts ?
##########
File path: website/www/site/content/en/documentation/transforms/java/elementwise/regex.md
##########
@@ -0,0 +1,34 @@
+---
+title: "Regex"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+# Regex
+<table align="left">
+ <a target="_blank" class="button"
+ href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/transforms/Regex.html">
+ <img src="https://beam.apache.org/images/logos/sdks/java.png" width="20px" height="20px"
+ alt="Javadoc" />
+ Javadoc
+ </a>
+</table>
+<br><br>
Review comment:
Why do we multiple breaks here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624337083
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-624884305
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bntnam edited a comment on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
bntnam edited a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620784079
@pabloem : Thanks for letting me know. I'll look into it. There is a commit which cleans Jekyll files, it causes conflicts. Probably, it stops you "retest this please". :D
And about the RAT PreCommit, do you know which files missing the Apache License? There are tons of files in the project, so idk which one it is. :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #11554: Website - Migrated Jekyll to Hugo
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-620839537
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia removed a comment on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia removed a comment on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625169190
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625344942
Run Website_Stage_GSC PreCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] mwalenia commented on pull request #11554: [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable localization of the site content
Posted by GitBox <gi...@apache.org>.
mwalenia commented on pull request #11554:
URL: https://github.com/apache/beam/pull/11554#issuecomment-625169459
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org