You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "bitsondatadev (via GitHub)" <gi...@apache.org> on 2023/09/26 21:05:27 UTC
[GitHub] [iceberg] bitsondatadev opened a new pull request, #8659: Phase 1: New Docs Deployment
bitsondatadev opened a new pull request, #8659:
URL: https://github.com/apache/iceberg/pull/8659
This PR is a culmination of my work in the [making-iceberg-docs](https://github.com/bitsondatadev/making-iceberg-docs) repository.
To see a preview, you can [check out the GitHub page](https://bitsondatadev.github.io/making-iceberg-docs/). We will eventually want to migrate this to using the iceberg GitHub pages to deploy the docs.
Design Discussion: https://lists.apache.org/thread/nq283g83p9cgx518jwo7dg85fm5gjrvv
Design Doc: https://docs.google.com/document/d/1WJXzcwC6isfoywcLY2lZ9gZN6i0JU1I2SIZmCkumbZc/edit#heading=h.gli9mc2ghfz1
Phase 1 Goals:
- Avoid changing the words and content themselves.
- Migrate away from Hugo front matter and syntax and migrate to [MkDocs-Material](https://squidfunk.github.io/mkdocs-material/) syntax.
- Fix broken links and verify using linkchecker.
- Add [Vale](https://vale.sh/) but initially disabled to hold off on doing too much.
- Combine the static (evergreen) pages and the versioned documentation into a single repository.
- Adjust the structure to match the layout of the website itself, where `./home/*.md` contains the evergreen content and `./home/docs/<version>/*.md` contains the versions of Iceberg.
- Keep all versioned javadocs in [their own tag](https://github.com/bitsondatadev/making-iceberg-docs/tree/javadoc).
- Keep each version of the versioned Iceberg docs in [their own tag](https://github.com/bitsondatadev/making-iceberg-docs/tree/docs-1.3.1) separate from the [equivalent release tag](https://github.com/apache/iceberg/tree/apache-iceberg-1.3.1) to enable future updates to older doc versions.
- Set up an all encompassing build using Spotify's [backstage/mkdocs-monorepo-plugin](https://github.com/backstage/mkdocs-monorepo-plugin) and [git worktrees](https://git-scm.com/docs/git-worktree) to build multiple sub-repositories to include the static root pages, the javadoc, and docs tags, and deploy these in a single site that is searched on.
- Ensure backwards compatibility with the existing site urls to avoid breaking permalinks.
Phase 2/3 and Todos:
- Style the webpage to match the existing <https://iceberg.apache.org>.
- Getting Started guide.
- Split content based on [diataxis framework](https://diataxis.fr/).
- Fix linting errors and add Vale to the build process.
- Add linkChecker to the build process.
- Move python docs to pyIceberg repo and build a submodule.
- Set up Rust/Go docs sites.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1741530893
RE: @RussellSpitzer cc: @ajantha-bhat
> How should we be reviewing this? What is new, what is old? Is it just style and build?
All of this is a complete copy of the words (no content changed) of the existing site. So I would not focus on the words. There were quite a few broken links on the site that I decided to fix and had to in order to enable the interlinking between the static docs and the versioned docs.
So, I would not be reviewing the actual words, or we'll be fixing the entire grammar and misspellings that already exist in the docs today. That will have it's own PR after this.
The best way to test this is to understand [the proposed build process and how it works](https://github.com/bitsondatadev/iceberg/tree/new-docs/docs-new#build-process), and [install it locally](https://github.com/bitsondatadev/iceberg/tree/new-docs/docs-new#install) to get a grasp of the mechanics involved in the deployment and see if you agree with it or have any concerns. You should also poke around a bit to verify I didn't break many permalinks.
This will not be deployed once merged, just give us an initial base for the new docs to work from. Later I will start working on the grammar fixes and adding [Vale, a prose linter](https://vale.sh/) that can abide by various grammar and style standards that we'd like to enforce.
Let me know if this makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1342179085
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
Sounds fine to me. I prefer to control ordering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367526033
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
+```
+
+## Build
+
+Run the build command in the root directory, and optionally add `--clean` to force MkDocs to clear previously generated pages.
+
+```
+mkdocs build [--clean]
+```
+
+## Run
+
+Start MkDocs server locally to verify the site looks good.
+
+```
+mkdocs serve
+```
+
+## Release process
+
+Deploying a version of the docs is a two step process:
+ 1. Cut a new release from the current branch revision. This creates a new branch `docs-<version>`,.
+
+ ```
+ .github/bin/deploy_docs.sh -v 1.4.0
Review Comment:
We should move this to maybe a `dev` folder with scripts rather than `.github` which seems like an odd place for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343936703
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
Well, actually site is generated in the same directory as the mkdocs folder, so we would need to rename the site directory. I actually like what Jekyll does and uses underscores to denote the special directories.
I'll use the `docs_dir: site` for the static pages and `site_dir: _site` for the generated pages.
To avoid nesting docs versioned sites by yet another layer, i'll keep `docs_dir: .` for the versioned docs but still use the `site_dir: _site` convention there as well. That will make it obvious that all `_site` directories are generated and to be ignored.
Let me know if you take any issue with this approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348077823
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
I don't understand what you're proposing, but I think that we should go with the defaults inside the mkdocs area. That is, `docs` should contain the source markdown files and is located in the same folder as the top-level `mkdocs.yaml`. `site` also lives in that folder when is it auto-generated. The folder containing all 3 can be called `site` or `docs`, I don't really have a preference there, but I'd lean toward sticking with what we have today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341782963
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
Review Comment:
Can you help me understand what's going on here? Why are the directories off by one level? What does that have to do with delayed aggregation, and what do you mean by delayed aggregation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341936383
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
Review Comment:
Agreed, but I wanted to do this in the next PR. Not this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338434476
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
> I can prioritize the migration to a separate repo.
I thought we were going to try and build this on separate ASF infra or something. Either way this would be good to get done, but I wouldn't prioritize it over the Arrow work. I'll need that for the `Getting Started` pages ;). Which I care about much more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1340867203
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
Review Comment:
That makes sense. Yeah, let's hit this deployment on the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on pull request #8659: Phase 1 - New Docs Deployment
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1741008270
How should we be reviewing this? What is new, what is old? Is it just style and build?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1342178885
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
Review Comment:
@bitsondatadev, I don't understand what sparse checkout has to do with branches vs tags.
According to the `git worktree` docs:
> To instead work on an existing branch in a new worktree, use `git worktree add <path> <branch>`
So you can use `git worktree` to check out an existing branch in a subdirectory. I think we should use that rather than tags because tags are hard to work with. Branches can be updated and we can open PRs against them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343891043
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
This wouldn't have to be now, but after we get this out add it to the roadmap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1774998904
@bitsondatadev Yes, I'm so excited to see this happening. Great work here! 🙌
I've created a project [here](https://github.com/orgs/apache/projects/300/views/1). Maybe jus a label might also suffice? I'll leave it up to ou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1775020283
Yeah, I think the projects make sense to have a better visual of progress broken down by phase. The first phase was a bunch of research and discussion on the design, so that would have been difficult to coordinate. The second phase is rather boring and though it is something that's easier to task out, my goal is to move quickly on that one so we don't have two versions of docs for long. The third phase is where I will build a bunch of tasks and get us to business as usual.
The second phase is updating older docs to the mkdocs format and tracking down a web dev to get mkdocs stylized to the current site. If anyone knows any open source web devs please let them know to reach out on the mailing list!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1340445915
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
Review Comment:
Alright, then you should not run the workflow on a `push` event on the main/master branch.
Then you want to replace it with:
```yaml
on:
tags:
- apache-iceberg-*
```
Also, in PyIceberg we have it as a [step in the release](https://py.iceberg.apache.org/how-to-release/#release-the-docs):
https://github.com/apache/iceberg/blob/eaf7c4f2e2adc952b5a150cf0b854adebfb9fdf0/.github/workflows/python-ci-docs.yml#L21-L22
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338406716
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
Review Comment:
lol oops :) yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341809008
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
+ - Spec: spec.md
Review Comment:
Should we have a "Specs" section?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341779867
##########
docs-new/home/ASF.md:
##########
@@ -0,0 +1,18 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+Apache Software Foundation links
Review Comment:
Is this missing content? Or was it just not accessible in the old site?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341936534
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
+ - Spec: spec.md
Review Comment:
Yes, but let's hold off these discussions for phase 2/3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341865071
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
Review Comment:
It's the directory layout of the docs so you can see the static docs versus the versioned docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341935660
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
I tried this, but there's no way to control the ordering. I did already have to hack monorepo to get the dot syntax in the url to work, perhaps we add that as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343407301
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
Review Comment:
Ah, I was originally trying to do this as an extension of the main release tags. When we realized we wanted the flexibility to update the older versions branches made more sense but for some reason I was thinking tags were a limitation of worktrees.
Anyways, yeah we can easily just do the same thing we're doing with tags and just update the branches that are rooted in the release tag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348198680
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
This causes warnings with the build expecting to see files in the relative locations across docs sites. I am using the current setup to try and minimize these errors so that it can be a notification that a link has broken.
Further, I'd like to continue moving towards a 1:1 relationship between the directory structure, and the sitemap, so newcomers can use the site layout to guide where they will find the file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348126895
##########
docs-new/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+#remove all local files and leave site/ dir
+rm ./*
Review Comment:
Won't remove directories without -R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348089146
##########
docs-new/README.md:
##########
@@ -0,0 +1,175 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This holds the static files that define and build the documentation site for Apache Iceberg.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./site/.*md` files and the docs are the `.site/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── site
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` branches that only contain the root of the docs version. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+└── site
+ ├── docs
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/1.3.1 docs-1.3.1
+git worktree add site/javadoc javadoc
+```
+
+## Build
+
+Run the build command in the root directory, and optionally add `--clean` to force MkDocs to clear previously generated pages.
+
+```
+mkdocs build [--clean]
+```
+
+## Run
+
+Start MkDocs server locally to verify the site looks good.
+
+```
+mkdocs serve
+```
+
+### Release process
+
+Deploying a version of the docs is a two step process:
+ 1. Cut a new release from the `latest` documentation which creates a new branch `docs-<version>`.
+
+ ```
+ .github/bin/deploy_docs.sh -v 1.4.0
+ ```
+
+ See [deploy_docs.sh](.github/bin/deploy_docs.sh) for more details.
+
+ 1. Make sure to add the new version to the list of versions to pull into git worktree.
+ 1. Follow the steps in [the build process](#build).
+ 1. Push the generated site to `gh-pages`.
Review Comment:
This will eventually need to be the ASF branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367525768
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
Review Comment:
These don't exist yet, so I had to comment out the 1.3.1 entry in mkdocs.yml to get `mkdocs serve` working. Not a blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367530281
##########
site/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+# remove all files and directories minus exceptions
+find . \
+ -not \( -path . \) \
+ -not \( -path .. \) \
+ -not \( -path ./.git -prune \) \
Review Comment:
Ah, so this actually runs in the root of the iceberg repo itself?
That's a bit of a problem because we won't want to run it locally. I would recommend updating this to not require deleting anything. Instead, I think this should copy what you need into folders and add just those folders.
To me, the steps should be:
1. Copy the `nightly` docs folder into a temp folder
2. Build the docs in the temp folder
3. Copy the contents needed out of the temp site folder
4. Check out an orphan branch (with no parent) using `--orphan`
5. Add the site folder to git
6. Commit the site folder
7. Switch back to the previous tag
That way, everything in the working copy is handled by git rather than by removing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367560477
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
Review Comment:
Added a warning to the README
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367530477
##########
site/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+# remove all files and directories minus exceptions
+find . \
+ -not \( -path . \) \
+ -not \( -path .. \) \
+ -not \( -path ./.git -prune \) \
+ -not \( -path ./.github -prune \) \
+ -not \( -path ./docs-new -prune \) \
+ ! -name LICENSE \
+ ! -name NOTICE \
+ -exec rm -rf {} +
+
+# move the nightly docs to the root and change from 'nightly'
+mv docs-new ./tmp && \
+mv ./tmp/docs/docs/nightly/* . && \
+rm -r ./tmp
+
+#delete top-level directories
+rm -r ./docs-new
+
+# update versions in mkdocs.yml
+sed -i '' -E "s/(^site\_name:[[:space:]]+docs\/).*$/\1${ICEBERG_VERSION}/" "./mkdocs.yml"
+sed -i '' -E "s/(^[[:space:]]*-[[:space:]]+Javadoc:.*\/javadoc\/)nightly/\1${ICEBERG_VERSION}/" "./mkdocs.yml"
+
+cd docs
+
+# add exclude search for older documentation
+python3 -c "import os
+for f in filter(lambda x: x.endswith('.md'), os.listdir()): lines = open(f).readlines(); open(f, 'w').writelines(lines[:2] + ['search:\n', ' exclude: true\n'] + lines[2:]);"
+
+cd ..
+
+git add .
Review Comment:
Adding specific folders would avoid the need to delete things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341780620
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
Review Comment:
Typo: contian.
Why use a tag for older javadoc instead of a branch? It seems odd to re-create a tag every time we need to change it instead of adding a new version of Javadoc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341776056
##########
docs-new/LICENSE:
##########
@@ -0,0 +1,201 @@
+ Apache License
Review Comment:
Why is there a LICENSE file? Wouldn't this be covered by the LICENSE in the root of the repository?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341809339
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
+ - Spec: spec.md
+ - View spec: view-spec.md
+ - Puffin spec: puffin-spec.md
+ - Multi-engine support: multi-engine-support.md
+ - How to release: how-to-release.md
+ - Terms: terms.md
+ - ASF:
+ - ASF: ASF.md
Review Comment:
I don't think we need this page. It doesn't have any content and wasn't in the [old mkdocs site](https://github.com/apache/iceberg/blob/apache-iceberg-0.13.2/site/mkdocs.yml).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341810046
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
Review Comment:
I think that this is missing a description of the process for releasing a new version.
If I understand correctly, we would take the `latest` docs that are in this repository and create a branch named `docs-<VERSION>` with just that folder, and then add the branch to the list of refs to pull in via git worktree?
A description of this process would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341866061
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
Review Comment:
Yes but just `cat ./link_warnings.csv`
The csv is just an output type specifier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341866343
##########
docs-new/home/roadmap.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Roadmap"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Roadmap Overview
+
+This roadmap outlines projects that the Iceberg community is working on, their priority, and a rough size estimate.
+This is based on the latest [community priority discussion](https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E).
+Each high-level item links to a Github project board that tracks the current status.
+Related design docs will be linked on the planning boards.
+
+# Priority 1
+
+* API: [Iceberg 1.0.0](https://github.com/apache/iceberg/projects/3) [medium]
Review Comment:
We haven't merged my PR that updated this yet. I need to get back to that one. Perhaps we can get it added to 1.4.0? Otherwise I'd like to avoid changing content yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347232147
##########
docs-new/site/releases.md:
##########
@@ -0,0 +1,777 @@
+---
+title: "Releases"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+## Downloads
+
+The latest version of Iceberg is [{{ icebergVersion }}](https://github.com/apache/iceberg/releases/tag/apache-iceberg-{{ icebergVersion }}).
+
+* [{{ icebergVersion }} source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.sha512)
+* [{{ icebergVersion }} Spark 3.4\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.12-{{ icebergVersion }}.jar) -- [3.4\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.3\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.12-{{ icebergVersion }}.jar) -- [3.3\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.2\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.12-{{ icebergVersion }}.jar) -- [3.2\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.1 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.1_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.1_2.12-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.17 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.17/{{ icebergVersion }}/iceberg-flink-runtime-1.17-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.16 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.16/{{ icebergVersion }}/iceberg-flink-runtime-1.16-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.15 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.15/{{ icebergVersion }}/iceberg-flink-runtime-1.15-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/{{ icebergVersion }}/iceberg-hive-runtime-{{ icebergVersion }}.jar)
+
+To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installation.
+
+To use Iceberg in Hive 2 or Hive 3, download the Hive runtime JAR and add it to Hive using `ADD JAR`.
+
+### Gradle
+
+To add a dependency on Iceberg in Gradle, add the following to `build.gradle`:
+
+```
+dependencies {
+ compile 'org.apache.iceberg:iceberg-core:{{ icebergVersion }}'
+}
+```
+
+You may also want to include `iceberg-parquet` for Parquet file support.
+
+### Maven
+
+To add a dependency on Iceberg in Maven, add the following to your `pom.xml`:
+
+```
+<dependencies>
+ ...
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-core</artifactId>
+ <version>{{ icebergVersion }}</version>
+ </dependency>
+ ...
+</dependencies>
+```
+
+## 1.3.1 release
+
+Apache Iceberg 1.3.1 was released on July 25, 2023.
+The 1.3.1 release addresses various issues identified in the 1.3.0 release.
+
+* Core
+ - Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots ([\#8064](https://github.com/apache/iceberg/pull/8064))
+* Hive
+ - Fix HiveCatalog deleting metadata on failures in checking lock status ([\#7931](https://github.com/apache/iceberg/pull/7931))
+* Spark
+ - Fix RewritePositionDeleteFiles failure for certain partition types ([\#8059](https://github.com/apache/iceberg/pull/8059))
+ - Fix RewriteDataFiles concurrency edge-case on commit timeouts ([\#7933](https://github.com/apache/iceberg/pull/7933))
+ - Fix partition-level DELETE operations for WAP branches ([\#7900](https://github.com/apache/iceberg/pull/7900))
+* Flink
+ - FlinkCatalog creation no longer creates the default database ([\#8039](https://github.com/apache/iceberg/pull/8039))
+
+## Past releases
+
+### 1.3.0 release
+
+Apache Iceberg 1.3.0 was released on May 30th, 2023.
+The 1.3.0 release adds a variety of new features and bug fixes.
+
+* Core
Review Comment:
Great catch @aokolnychyi!
Looks like the markdown Hugo uses isn't as picky with indentation.
Here's the 1.3.1 release markdown.
```
* Core
- Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots ([\#8064](https://github.com/apache/iceberg/pull/8064))
* Hive
- Fix HiveCatalog deleting metadata on failures in checking lock status ([\#7931](https://github.com/apache/iceberg/pull/7931))
```
Which [results in what you've shown above](https://bitsondatadev.github.io/making-iceberg-docs/releases/#131-release).
I checked if there was a change after we moved away from mkdocs previously and keeps rendering incorrectly through 0.13.0:
```
**High-level features:**
* **Core**
* Catalog caching now supports cache expiration through catalog property `cache.expiration-interval-ms` [[\#3543](https://github.com/apache/iceberg/pull/3543)]
```
Which still [renders incorrectly](https://bitsondatadev.github.io/making-iceberg-docs/releases/#0130).
0.12.0 on the other hand, renders correctly and it seems the indentation fixes it.
```
**High-level features:**
* **Core**
* Allow Iceberg schemas to specify one or more columns as row identifiers [[\#2465](https://github.com/apache/iceberg/pull/2465)]. Note that this is a prerequisite for supporting upserts in Flink.
```
Which [renders what I believe is expected](https://bitsondatadev.github.io/making-iceberg-docs/releases/#0120).
I would propose instead that we follow the 0.14.0 release format that makes the section title H4 elements.
```
### 0.14.0 release
Apache Iceberg 0.14.0 was released on 16 July 2022.
#### Highlights
* Added several [performance improvements](#performance-improvements) for scan planning and Spark queries
* Added a common REST catalog client that uses change-based commits to resolve commit conflicts on the service side
```
Which [renders properly](https://bitsondatadev.github.io/making-iceberg-docs/releases/#0140-release) as it is standard compatible markdown. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1747804852
> My primary concern of moving the docs into the main repo was versioning and pollution. It seems like `git worktree` should solve that. I deployed this locally, it seems pretty straightforward.
>
>
>
> Do we need to adjust our `.gitignore`? I did see some extra files when deploying locally.
There's still a few changes I need to make that I missed when adapting this from my sandbox repo.
My goal is to not have any artifacts like this remaining once this is merged. I want this to be a good baseline to generate the older versions of the docs with the consistent mkdocs format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1340440838
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
> Totally agree, I think this won't be able to be applied in this initial PR though as I'm methodically trying to split up the work and this option would increase the scope of Phase 1: https://github.com/apache/iceberg/pull/8659#issuecomment-1737155953.
Yes, I wasn't suggesting to do it in this PR, but I just wanted to plant the seed.
> I thought we were going to try and build this on separate ASF infra or something. Either way this would be good to get done, but I wouldn't prioritize it over the Arrow work. I'll need that for the Getting Started pages ;). Which I care about much more.
With the current CI that you have in this repository, it will [overwrite the `gh-pages` branch](https://github.com/apache/iceberg/tree/gh-pages), and therefore whipping the PyIceberg docs. We can also deploy to the [ASF infra as we do with iceberg-docs](https://github.com/apache/iceberg-docs/tree/asf-site), but that requires something different than the `mkdocs gh-deploy` that's in there now. For reference: [docs on gh-deploy](https://www.mkdocs.org/user-guide/deploying-your-docs/#project-pages).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341784679
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
+```
+
+## Things to consider
+
+ - Do not use static links to the public iceberg site.
Review Comment:
Do you mean "to" or "in"? I think it's fine to link TO the Iceberg site from others. Within the site, though, we only want to use relative links?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341792493
##########
docs-new/home/index.md:
##########
@@ -0,0 +1,108 @@
+---
+title: "Intro to Apache Iceberg"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+<link href="https://cdn.jsdelivr.net/gh/apache/iceberg-docs@main/iceberg-theme/static/css/termynal.css" rel="stylesheet">
+<style>
+section#intro {
+ display: block;
+ max-width: 100%;
+ margin: 0;
+ padding: 0;
+}
+
+.intro-header {
+ background: url(assets/images/intro-bg.webp) no-repeat center center;
+ background-size: cover;
+}
+</style>
+
+
+<section id="intro">
+ <div class="intro-header">
+ </div>
+</section>
+
+## Apache Iceberg
+
+### The open table format for analytic datasets.
+
+* [Community](community.md)
+* [GitHub](https://github.com/apache/iceberg)
+* [Slack](https://github.com/apache/iceberg)
+
+## What is Iceberg?</h2>
Review Comment:
Looks like an unnecessary `</h2>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341864902
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
Review Comment:
Good question. It's complicated.
First, docs url path and javadocs url path are in different locations, so they each have their own mount path when considering we use git worktree to pull the different versions together at build time.
So the git worktree will mount whatever is in the tag at a given subdirectory. My initial plan was actually to do git worktree and [git sparse checkout](https://git-scm.com/docs/git-sparse-checkout) to define specifically what sections of the tag or remote tag I needed and place them in the proper points in the docs. This would allow me the flexibility to just define a tag per version that includes Java docs and Iceberg versioned docs. However, I was unable to get sparse checkout to work properly with worktree and I also noticed this in the docs:
> THIS COMMAND IS EXPERIMENTAL. ITS BEHAVIOR, AND THE BEHAVIOR OF OTHER COMMANDS IN THE PRESENCE OF SPARSE-CHECKOUTS, WILL LIKELY CHANGE IN THE FUTURE.
So instead I need the tags to contain the exact directory at the root that needs to be mounted for the docs. So that's what my deploy script handles.
Now there's some debate we can have on how we want to do this.
- If someone is a git wizard and can help me understand the behavior of worktree in tandem with sparse checkout, I'm happy to dig back into that if we think the future changes won't be disruptive but I doubt that's the path we want to take.
- We can also consider sticking all docs and Javadocs into a single tag and just update that tag.
- We could pair docs and javadocs into a tag together and every time I add a worktree, I'll have another part of the script that moves javadocs to the correct location.
- Just do what we're doing now.
I'm fine with any of these approaches really. I just chose the current one thinking it would make things easier to build from older versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1737155953
> Great work @bitsondatadev
>
> Looks really nice and shiny! The layout reminds me of [another side](https://py.iceberg.apache.org/) that I like. Left a few comments and pointers, my main concern is that we don't clearly state the version (or dev) on the side, so it is not obvious to a user which version they are looking at.
Thanks @Fokko!
You touched on a few points I missed around my intended next steps.
Finishing Phase 1:
* Once 1.4.0 is released, cherry-pick/merge the changes between docs and build out all the release tags from 1.4.0 back to the earliest documentation versions we have.
* Work with Fokko/ASF INFRA to get a preview site deployed (that doesn't step on any other existing site's urls.
* Initially merge this under the docs_new folder to complete this first round of work.
Phase 2:
* Run a few cleanup PRs that address strictness links and build process. This will include adding and applying vale rules, linkchecker to the build, getting mkdocs to compile docs without any error, fixing search index around 'spark' keyword, working on version labels (dev) etc...
* Simultaneously, fix the look and feel of these sites to move to the look and feel of the existing site.
* Finally, once the links/syntax are all in check, replace docs/ with new_docs/ and move the site over to build in iceberg.apache.org domain.
Phase 3: Will then be addressing the organization of the site (which is outlined under "Phase 2" in the design docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338422742
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
Totally agree, I think this won't be able to be applied in this initial PR though as I'm methodically trying to split up the work and this option would increase the scope of Phase 1: [See my comment](https://github.com/apache/iceberg/pull/8659#issuecomment-1737155953).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341799374
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
I think that `latest` should always be the latest release, not the state of the repository. I'd recommend adding `nightly` for this content.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367530603
##########
site/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+# remove all files and directories minus exceptions
+find . \
+ -not \( -path . \) \
+ -not \( -path .. \) \
+ -not \( -path ./.git -prune \) \
+ -not \( -path ./.github -prune \) \
+ -not \( -path ./docs-new -prune \) \
+ ! -name LICENSE \
+ ! -name NOTICE \
+ -exec rm -rf {} +
+
+# move the nightly docs to the root and change from 'nightly'
+mv docs-new ./tmp && \
+mv ./tmp/docs/docs/nightly/* . && \
+rm -r ./tmp
+
+#delete top-level directories
+rm -r ./docs-new
+
+# update versions in mkdocs.yml
+sed -i '' -E "s/(^site\_name:[[:space:]]+docs\/).*$/\1${ICEBERG_VERSION}/" "./mkdocs.yml"
+sed -i '' -E "s/(^[[:space:]]*-[[:space:]]+Javadoc:.*\/javadoc\/)nightly/\1${ICEBERG_VERSION}/" "./mkdocs.yml"
+
+cd docs
+
+# add exclude search for older documentation
+python3 -c "import os
+for f in filter(lambda x: x.endswith('.md'), os.listdir()): lines = open(f).readlines(); open(f, 'w').writelines(lines[:2] + ['search:\n', ' exclude: true\n'] + lines[2:]);"
+
+cd ..
+
+git add .
+
+git commit -m "Deploy ${GIT_BRANCH} branch"
+
+git push origin $GIT_BRANCH
Review Comment:
`$REMOTE` rather than `origin`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341798162
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
Review Comment:
Spark should come first. I think the only reason why it didn't in the old site was that Hugo was ordering alphabetically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341886382
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
+```
+
+## Things to consider
+
+ - Do not use static links to the public iceberg site.
Review Comment:
Correct we only want relative links and we don't want to assume we're in the public Apache website.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341885827
##########
docs-new/variables.yml:
##########
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_url: http://localhost:8000
Review Comment:
Ah this needs to be ripped out anyways, I was hoping to avoid relative links by specifying absolute links from an adjustable root url but mkdocs only works with relative links.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341936676
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
+ - Spec: spec.md
+ - View spec: view-spec.md
+ - Puffin spec: puffin-spec.md
+ - Multi-engine support: multi-engine-support.md
+ - How to release: how-to-release.md
+ - Terms: terms.md
+ - ASF:
+ - ASF: ASF.md
Review Comment:
Yeah, I'll play with this. I forgot I had this here and mkdocs was doing something funky with the lack of any index md file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341779255
##########
docs-new/home/assets/images/intro-bg.jpg:
##########
Review Comment:
Same with this one. We can remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341783991
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
Review Comment:
Minor: We avoid using personal pronouns like "I" or "we" in documentation (like this README). I think this is intended to explain what "you" are "currently" doing, but when this is committed it will be documentation for building and maintaining the site.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1342179203
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
Yeah, we'd have site/site. I think that's fine and probably better than `home` because people don't care about generated paths. Where they go looking for docs to change is more important.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338430425
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
Review Comment:
> This would mean that it shows features that haven't been released yet.
Not on the live site. I assumed that this build would only trigger on a release, so (latest) and the current version (e.g. 1.3.1 today) would just be duplicate docs (Unfortunately redirects from `latest` > `<version>` that maintain the `/docs/latest/` url would require some fancy DNS records that @rdblue said we should avoid for now).
Later, we should defnitely consider adding a (nightly) or (dev) build and we would label that as such. For now, just assume that we only build this during a release. Unless the concern is around when people build the site locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1737244587
@ajantha-bhat,what level of granularity do you suggest for the refactoring?
If you want me to treat the static content and versioned docs as separate changes, I don't see much value there. The hard part about this phase was making a single build process that could stitch multiple versioned docs and static docs into a single build. I need both of these elements to create such a system and showcase it to the community. There is little value in just copying the iceberg-docs over in its own PR IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1349336499
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
What do you suggest when you say "point" a 307 (temp) redirect?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1771793120
@rdblue I have this now in a good state relative to the 1.4.0 docs in nightly, could you PTAL? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367526487
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
+```
+
+## Build
+
+Run the build command in the root directory, and optionally add `--clean` to force MkDocs to clear previously generated pages.
Review Comment:
By "root directory" I assume you mean `site/` because that's where I'm running it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1774978383
> @bitsondatadev have you created a project to track progress and to-do list? Maybe others can help and pick up some tasks as well.
I want to but I don't have the ability to set it up. Seeing that @Fokko added an emoji response maybe he can help me with this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347234000
##########
docs-new/site/releases.md:
##########
@@ -0,0 +1,777 @@
+---
+title: "Releases"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+## Downloads
+
+The latest version of Iceberg is [{{ icebergVersion }}](https://github.com/apache/iceberg/releases/tag/apache-iceberg-{{ icebergVersion }}).
+
+* [{{ icebergVersion }} source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.sha512)
+* [{{ icebergVersion }} Spark 3.4\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.12-{{ icebergVersion }}.jar) -- [3.4\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.3\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.12-{{ icebergVersion }}.jar) -- [3.3\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.2\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.12-{{ icebergVersion }}.jar) -- [3.2\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.1 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.1_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.1_2.12-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.17 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.17/{{ icebergVersion }}/iceberg-flink-runtime-1.17-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.16 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.16/{{ icebergVersion }}/iceberg-flink-runtime-1.16-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.15 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.15/{{ icebergVersion }}/iceberg-flink-runtime-1.15-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/{{ icebergVersion }}/iceberg-hive-runtime-{{ icebergVersion }}.jar)
+
+To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installation.
+
+To use Iceberg in Hive 2 or Hive 3, download the Hive runtime JAR and add it to Hive using `ADD JAR`.
+
+### Gradle
+
+To add a dependency on Iceberg in Gradle, add the following to `build.gradle`:
+
+```
+dependencies {
+ compile 'org.apache.iceberg:iceberg-core:{{ icebergVersion }}'
+}
+```
+
+You may also want to include `iceberg-parquet` for Parquet file support.
+
+### Maven
+
+To add a dependency on Iceberg in Maven, add the following to your `pom.xml`:
+
+```
+<dependencies>
+ ...
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-core</artifactId>
+ <version>{{ icebergVersion }}</version>
+ </dependency>
+ ...
+</dependencies>
+```
+
+## 1.3.1 release
+
+Apache Iceberg 1.3.1 was released on July 25, 2023.
+The 1.3.1 release addresses various issues identified in the 1.3.0 release.
+
+* Core
+ - Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots ([\#8064](https://github.com/apache/iceberg/pull/8064))
+* Hive
+ - Fix HiveCatalog deleting metadata on failures in checking lock status ([\#7931](https://github.com/apache/iceberg/pull/7931))
+* Spark
+ - Fix RewritePositionDeleteFiles failure for certain partition types ([\#8059](https://github.com/apache/iceberg/pull/8059))
+ - Fix RewriteDataFiles concurrency edge-case on commit timeouts ([\#7933](https://github.com/apache/iceberg/pull/7933))
+ - Fix partition-level DELETE operations for WAP branches ([\#7900](https://github.com/apache/iceberg/pull/7900))
+* Flink
+ - FlinkCatalog creation no longer creates the default database ([\#8039](https://github.com/apache/iceberg/pull/8039))
+
+## Past releases
+
+### 1.3.0 release
+
+Apache Iceberg 1.3.0 was released on May 30th, 2023.
+The 1.3.0 release adds a variety of new features and bug fixes.
+
+* Core
Review Comment:
Also, this will wait until the next phase. There are a lot of issues like this that I'd like to address in a separate PR.
I only addressed making links consistent in this PR because that was required to get internal links across the top-level and versioned sites working correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348043125
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
+ - Spec: spec.md
+ - View spec: view-spec.md
+ - Puffin spec: puffin-spec.md
+ - Multi-engine support: multi-engine-support.md
+ - How to release: how-to-release.md
+ - Terms: terms.md
+ - ASF:
+ - ASF: ASF.md
Review Comment:
So ultimately there's an issue with how mkdocs renders.
https://github.com/squidfunk/mkdocs-material/issues/868
The solution is to provide an [override](https://squidfunk.github.io/mkdocs-material/customization/#overriding-partials) to the materials theme. I'm going to dig into that later though. I've removed the ASF.md file for now. The current functionality is that the "ASF" nav link just links directly to "Sponsorship" (the first sub nav item in the index) so for now I'm just linking to community.md so that we'll at least be able to open a local page to see the nav.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1749722304
I've addressed the comments I intend to hit this review round, but have some open questions:
- https://github.com/apache/iceberg/pull/8659#discussion_r1343890396
- https://github.com/apache/iceberg/pull/8659#discussion_r1341799374
- https://github.com/apache/iceberg/pull/8659#discussion_r1341800289
Anton:
- https://github.com/apache/iceberg/pull/8659#issuecomment-1747804852
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348077823
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
I don't understand what you're proposing, but I think that we should go with the defaults inside the mkdocs area. That is, `docs` should contain the source markdown files and is located in the same folder as the top-level `mkdocs.yaml`. `site` also lives in that folder when is it auto-generated. The folder containing all 3 can be called `site` or `docs`, I don't really have a preference there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343936703
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
Well, actually site is generated in the same directory as the mkdocs folder, so we would need to rename the site directory. I actually like what Jekyll does and uses underscores to denote Jekyll specific directories.
I'll use the `docs_dir: site` for the static pages and `site_dir: _site` for the generated pages.
To avoid nesting docs versioned sites by yet another layer, i'll keep `docs_dir: .` for the versioned docs but still use the `site_dir: _site` convention there as well. That will make it obvious that all `_site` directories are generated and to be ignored.
Let me know if you take any issue with this approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1346544156
##########
docs-new/site/releases.md:
##########
@@ -0,0 +1,777 @@
+---
+title: "Releases"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+## Downloads
+
+The latest version of Iceberg is [{{ icebergVersion }}](https://github.com/apache/iceberg/releases/tag/apache-iceberg-{{ icebergVersion }}).
+
+* [{{ icebergVersion }} source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.sha512)
+* [{{ icebergVersion }} Spark 3.4\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.12-{{ icebergVersion }}.jar) -- [3.4\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.3\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.12-{{ icebergVersion }}.jar) -- [3.3\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.2\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.12-{{ icebergVersion }}.jar) -- [3.2\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.1 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.1_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.1_2.12-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.17 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.17/{{ icebergVersion }}/iceberg-flink-runtime-1.17-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.16 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.16/{{ icebergVersion }}/iceberg-flink-runtime-1.16-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.15 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.15/{{ icebergVersion }}/iceberg-flink-runtime-1.15-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/{{ icebergVersion }}/iceberg-hive-runtime-{{ icebergVersion }}.jar)
+
+To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installation.
+
+To use Iceberg in Hive 2 or Hive 3, download the Hive runtime JAR and add it to Hive using `ADD JAR`.
+
+### Gradle
+
+To add a dependency on Iceberg in Gradle, add the following to `build.gradle`:
+
+```
+dependencies {
+ compile 'org.apache.iceberg:iceberg-core:{{ icebergVersion }}'
+}
+```
+
+You may also want to include `iceberg-parquet` for Parquet file support.
+
+### Maven
+
+To add a dependency on Iceberg in Maven, add the following to your `pom.xml`:
+
+```
+<dependencies>
+ ...
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-core</artifactId>
+ <version>{{ icebergVersion }}</version>
+ </dependency>
+ ...
+</dependencies>
+```
+
+## 1.3.1 release
+
+Apache Iceberg 1.3.1 was released on July 25, 2023.
+The 1.3.1 release addresses various issues identified in the 1.3.0 release.
+
+* Core
+ - Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots ([\#8064](https://github.com/apache/iceberg/pull/8064))
+* Hive
+ - Fix HiveCatalog deleting metadata on failures in checking lock status ([\#7931](https://github.com/apache/iceberg/pull/7931))
+* Spark
+ - Fix RewritePositionDeleteFiles failure for certain partition types ([\#8059](https://github.com/apache/iceberg/pull/8059))
+ - Fix RewriteDataFiles concurrency edge-case on commit timeouts ([\#7933](https://github.com/apache/iceberg/pull/7933))
+ - Fix partition-level DELETE operations for WAP branches ([\#7900](https://github.com/apache/iceberg/pull/7900))
+* Flink
+ - FlinkCatalog creation no longer creates the default database ([\#8039](https://github.com/apache/iceberg/pull/8039))
+
+## Past releases
+
+### 1.3.0 release
+
+Apache Iceberg 1.3.0 was released on May 30th, 2023.
+The 1.3.0 release adds a variety of new features and bug fixes.
+
+* Core
Review Comment:
I think this is not formatted correctly when I deploy locally.
<img width="732" alt="image" src="https://github.com/apache/iceberg/assets/6235869/fd2ea1b7-d9c2-432b-81ca-0bc96b0bd762">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367559085
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
+```
+
+## Build
+
+Run the build command in the root directory, and optionally add `--clean` to force MkDocs to clear previously generated pages.
Review Comment:
Moving forward should this be built from `./site/` then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1774333581
@bitsondatadev have you created a project to track progress and to-do list? Maybe others can help and pick up some tasks as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367527441
##########
site/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+# remove all files and directories minus exceptions
+find . \
+ -not \( -path . \) \
+ -not \( -path .. \) \
+ -not \( -path ./.git -prune \) \
+ -not \( -path ./.github -prune \) \
+ -not \( -path ./docs-new -prune \) \
+ ! -name LICENSE \
+ ! -name NOTICE \
+ -exec rm -rf {} +
Review Comment:
Does this require `-f`? That's not a good thing to have in scripts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341788832
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
Review Comment:
It seems like the documentation here is necessary to understand what's going on: https://github.com/backstage/mkdocs-monorepo-plugin#usage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341777883
##########
docs-new/.vale.ini:
##########
@@ -0,0 +1,48 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+StylesPath = .github/vale/styles
+
+Vocab = iceberg
+
+MinAlertLevel = suggestion
+
+Packages = Microsoft, alex
+
+[*]
+BasedOnStyles = Vale
+TokenIgnores = \{\{\s*[^\n$]+\s*\}\}
Review Comment:
If Vale isn't turned on, I'd probably remove it from this PR to avoid over-complicating it. Otherwise we won't review this and will commit unreviewed code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1349160440
##########
docs-new/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+#remove all local files and leave site/ dir
+rm ./*
Review Comment:
Is it leaving all the directories then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue merged PR #8659:
URL: https://github.com/apache/iceberg/pull/8659
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367561171
##########
site/README.md:
##########
@@ -0,0 +1,182 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Iceberg site and documentation
+
+This subproject contains the [MkDocs projects](https://www.mkdocs.org/) that define the non-versioned Iceberg site and the versioned Iceberg documentation. The Iceberg site MkDocs project contains a plugin that builds all the static sites for each version of documentation. These subsites are all weaved together to make the final Apache Iceberg site and docs with a single build.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Usage
+
+The directory structure in this repository mimics the sitemap hierarchy of the website. This aims to help contributors find the source files needed to make their changes faster. To understand the layout and naming, it is helpful to have some basic understandings of the MkDocs framework defaults.
+
+### MkDocs background
+
+In MkDocs, the [`docs_dir`](https://www.mkdocs.org/user-guide/configuration/#docs_dir) points to the root directory containing the source markdown files for an MkDocs project. By default, this points to the `docs` directory. When you build MkDocs `mkdocs build`, MkDocs generates the static site in the [`site_dir`](https://www.mkdocs.org/user-guide/configuration/#site_dir) becomes the root of that project for the generated site.
+
+### Iceberg docs layout
+
+In the Iceberg docs, since the top-level site and versioned docs are contained in the same directory, they all live under the `/site` directory of the main Iceberg repository. The `/site/docs` directory is named this way to follow the [MkDocs convention](https://www.mkdocs.org/user-guide/configuration/#docs_dir), while the `/site/docs/docs` directory is an analog to the "Docs" navigation tab. Under this directory, you'll find the `/site/docs/docs/nightly` directory, which contains the state of the documentation in the local revisions.
+
+The non-versioned site pages are all the `/site/docs/.*md` files and the docs are the `/site/docs/docs/<version>/docs/*.md` files. Notice the location of the `mkdocs.yml`. Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+./site/
+├── docs
+│ ├── assets
+│ ├── docs
+│ │ └── nightly
+│ │ ├── docs
+│ │ │ ├── assets
+│ │ │ ├── api.md
+│ │ │ ├── ...
+│ │ │ └── table-migration.md
+│ │ └── mkdocs.yml (versioned)
+│ ├── about.md
+│ ├── ...
+│ └── view-spec.md
+├── README.md
+├── mkdocs.yml (non-versioned)
+├── requirements.txt
+└── variables.yml
+```
+### Building the versioned docs
+
+All previously versioned docs will be committed in `docs-<version>` branches and mounted using [git worktree](https://git-scm.com/docs/git-worktree) at build time. The worktree will pull these versions in following the `/site/docs/docs/<version>` convention. The `latest` version, will be a secondary copy of the most recent build version in the worktree, but pointing to `/site/docs/docs/latest`. There is also a `javadoc` branch that contains all prior static generation versions of the javadocs in a single tag.
+
+```
+./site/
+└── docs
+ ├── docs
+ │ ├── nightly
+ │ ├── latest
+ │ ├── 1.3.1
+ │ ├── 1.3.0
+ │ └── ...
+ └── javadoc
+ ├── latest
+ ├── 1.3.1
+ ├── 1.3.0
+ └── ...
+```
+
+### Install
+
+1. (Optional) Set up venv
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+```
+
+1. Install required Python libraries
+```
+pip install -r requirements.txt
+```
+
+#### Adding additional versioned documentation
+
+To build locally with additional docs versions, add them to your working tree.
+For now, I'm just adding a single version, and the javadocs directory.
+
+```
+git worktree add site/docs/docs/1.3.1 docs-1.3.1
+git worktree add site/docs/javadoc javadoc
+```
+
+## Build
+
+Run the build command in the root directory, and optionally add `--clean` to force MkDocs to clear previously generated pages.
+
+```
+mkdocs build [--clean]
+```
+
+## Run
+
+Start MkDocs server locally to verify the site looks good.
+
+```
+mkdocs serve
+```
+
+## Release process
+
+Deploying a version of the docs is a two step process:
+ 1. Cut a new release from the current branch revision. This creates a new branch `docs-<version>`,.
+
+ ```
+ .github/bin/deploy_docs.sh -v 1.4.0
Review Comment:
Added a note here as well to ignore while we remove the script.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367527689
##########
site/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,57 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+# remove all files and directories minus exceptions
+find . \
+ -not \( -path . \) \
+ -not \( -path .. \) \
+ -not \( -path ./.git -prune \) \
+ -not \( -path ./.github -prune \) \
+ -not \( -path ./docs-new -prune \) \
+ ! -name LICENSE \
+ ! -name NOTICE \
+ -exec rm -rf {} +
+
+# move the nightly docs to the root and change from 'nightly'
+mv docs-new ./tmp && \
Review Comment:
I think this needs to be updated to be `site`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1367527094
##########
site/.github/bin/deploy_docs.sh.orig:
##########
@@ -0,0 +1,67 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+cd ..
+
+<<<<<<< HEAD
Review Comment:
Looks like a bad merge to me. This file shouldn't be here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338430425
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
Review Comment:
> This would mean that it shows features that haven't been released yet.
Not on the live site. I assumed that this build would only trigger on a release, so (latest) and the current version (e.g. 1.3.1 today) would just be duplicate docs (Unfortunately redirects from `latest` > `<version>` that maintain the `/docs/latest/` url would require some fancy DNS records that @rdblue said we should avoid for now).
Later, we should defnitely consider adding a (nightly) or (dev) build and we would label that as such. For now, just assume that we only build this during a release. Unless the concern is around when people build the site locally.
@nastra also brought this point up to me directly but I forgot to add this in the design so thanks for bringing this up so I can explain it here!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1737199784
@bitsondatadev: Is it possible to split into two PRs?
One is to move back the existing docs from iceberg-docs repo and another one is for refactoring?
It may be easy to review the incremental change for the second PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341935200
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
+```
+
+## Things to consider
+
+ - Do not use static links to the public iceberg site.
+ - Only use relative links. If you want to reference the root (the directory where the main mkdocs.yml is located `home` in our case) use "spec.md" vs "/spec.md". Also, static sites should only reference the `docs/*` (see next point), but docs can reference the static content normally (e.g. `branching.md` page which is a versioned page linking to `spec.md` which is a static page).
Review Comment:
Yeah, static pages can link down to docs fine, it's linking back up that requires "../../spec.md" vs "spec.md".
Unfortunately the way you link is dependent on the context of which doc you are writing in. I tried to make this consistent across all areas of the project, but couldn't find a solution that worked with monorepo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341935924
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
Sure, but we would need to add and maintain a redirect for every single page from latest to the current version. I actually think this could be generated but this continues to add more complexity to our build. Something to consider though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341787262
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
+```
+
+## Things to consider
+
+ - Do not use static links to the public iceberg site.
+ - Only use relative links. If you want to reference the root (the directory where the main mkdocs.yml is located `home` in our case) use "spec.md" vs "/spec.md". Also, static sites should only reference the `docs/*` (see next point), but docs can reference the static content normally (e.g. `branching.md` page which is a versioned page linking to `spec.md` which is a static page).
Review Comment:
Can the static pages link to docs in `docs/latest`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341793838
##########
docs-new/variables.yml:
##########
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_url: http://localhost:8000
Review Comment:
Is this correct? Looks like it is for local testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1741516820
> @bitsondatadev: Is it possible to split into two PRs?
> One is to move back the existing docs from iceberg-docs repo and another one is for refactoring?
> It may be easy to review the incremental change for the second PR.
I don't think this is a good idea. We don't want to add the iceberg-docs repo here, it's just unnecessary churn. I think the motivation is to be able to see what's changing, but that's fairly straightforward since this is not changing the content. Only the site structure and build are changing. The pages should say exactly the same thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1342179144
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
I don't think that we need redirects. We don't have them now. We just have two copies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1349601992
##########
docs-new/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+#remove all local files and leave site/ dir
+rm ./*
Review Comment:
Yes, and I delete those a few lines below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1347234000
##########
docs-new/site/releases.md:
##########
@@ -0,0 +1,777 @@
+---
+title: "Releases"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+## Downloads
+
+The latest version of Iceberg is [{{ icebergVersion }}](https://github.com/apache/iceberg/releases/tag/apache-iceberg-{{ icebergVersion }}).
+
+* [{{ icebergVersion }} source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-{{ icebergVersion }}/apache-iceberg-{{ icebergVersion }}.tar.gz.sha512)
+* [{{ icebergVersion }} Spark 3.4\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.12-{{ icebergVersion }}.jar) -- [3.4\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.4_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.4_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.3\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.12-{{ icebergVersion }}.jar) -- [3.3\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.3_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.3_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.2\_2.12 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.12-{{ icebergVersion }}.jar) -- [3.2\_2.13](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.13/{{ icebergVersion }}/iceberg-spark-runtime-3.2_2.13-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Spark 3.1 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.1_2.12/{{ icebergVersion }}/iceberg-spark-runtime-3.1_2.12-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.17 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.17/{{ icebergVersion }}/iceberg-flink-runtime-1.17-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.16 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.16/{{ icebergVersion }}/iceberg-flink-runtime-1.16-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Flink 1.15 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime-1.15/{{ icebergVersion }}/iceberg-flink-runtime-1.15-{{ icebergVersion }}.jar)
+* [{{ icebergVersion }} Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/{{ icebergVersion }}/iceberg-hive-runtime-{{ icebergVersion }}.jar)
+
+To use Iceberg in Spark or Flink, download the runtime JAR for your engine version and add it to the jars folder of your installation.
+
+To use Iceberg in Hive 2 or Hive 3, download the Hive runtime JAR and add it to Hive using `ADD JAR`.
+
+### Gradle
+
+To add a dependency on Iceberg in Gradle, add the following to `build.gradle`:
+
+```
+dependencies {
+ compile 'org.apache.iceberg:iceberg-core:{{ icebergVersion }}'
+}
+```
+
+You may also want to include `iceberg-parquet` for Parquet file support.
+
+### Maven
+
+To add a dependency on Iceberg in Maven, add the following to your `pom.xml`:
+
+```
+<dependencies>
+ ...
+ <dependency>
+ <groupId>org.apache.iceberg</groupId>
+ <artifactId>iceberg-core</artifactId>
+ <version>{{ icebergVersion }}</version>
+ </dependency>
+ ...
+</dependencies>
+```
+
+## 1.3.1 release
+
+Apache Iceberg 1.3.1 was released on July 25, 2023.
+The 1.3.1 release addresses various issues identified in the 1.3.0 release.
+
+* Core
+ - Table Metadata parser now accepts null for fields: current-snapshot-id, properties, and snapshots ([\#8064](https://github.com/apache/iceberg/pull/8064))
+* Hive
+ - Fix HiveCatalog deleting metadata on failures in checking lock status ([\#7931](https://github.com/apache/iceberg/pull/7931))
+* Spark
+ - Fix RewritePositionDeleteFiles failure for certain partition types ([\#8059](https://github.com/apache/iceberg/pull/8059))
+ - Fix RewriteDataFiles concurrency edge-case on commit timeouts ([\#7933](https://github.com/apache/iceberg/pull/7933))
+ - Fix partition-level DELETE operations for WAP branches ([\#7900](https://github.com/apache/iceberg/pull/7900))
+* Flink
+ - FlinkCatalog creation no longer creates the default database ([\#8039](https://github.com/apache/iceberg/pull/8039))
+
+## Past releases
+
+### 1.3.0 release
+
+Apache Iceberg 1.3.0 was released on May 30th, 2023.
+The 1.3.0 release adds a variety of new features and bug fixes.
+
+* Core
Review Comment:
Also, this will wait until the next phase. There are a lot of issues like this that I'd like to address in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1342178970
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
Review Comment:
Sounds like more reason to show the command to view the output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1740316097
Hey @ajantha-bhat @Fokko, I've now updated the PR to remove the unecessary static files which took away more than half of the files so this should be much easier to look at.
For the rest of it, I explain the layout and rationale [a bit more in the README](https://github.com/bitsondatadev/iceberg/tree/new-docs/docs-new#build-process) now around the build process and what we are trying to achieve.
If you have any qeustions do let me know please! I think this is now in a state where I feel good sharing on the mailing list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #8659: Phase 1 - New Docs Deployment
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1741029866
> How should we be reviewing this? What is new, what is old? Is it just style and build?
That was my intention of asking https://github.com/apache/iceberg/pull/8659#issuecomment-1737199784 too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341808861
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
+# - 1.3.0: '!include home/docs/1.3.0/mkdocs.yml'
+ - Releases: releases.md
+ - Roadmap: roadmap.md
+ - Blogs: blogs.md
+ - Talks: talks.md
+ - Vendors: vendors.md
+ - Project:
+ - Join: community.md
Review Comment:
Let's name this back to "Community". I've never understood why it was renamed :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341798674
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
We may be able to use the `*include` syntax here. There's an example in the mkdocs-monorepo-plugin README: `'*include ./components/*/mkdocs.yml'`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341800289
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
Is there a more obvious name? I think most people expect `docs` for mkdocs projects but I see why you'd want to change it so we don't have `docs/docs`. Before, we had a top-level `site` folder and left this as `docs`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341781896
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
Review Comment:
Can you explain why we would run these commands? I'd expect something like "To build locally with additional docs versions, add them to your working tree like this ..."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341784233
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
+.
+├── home
+│ ├── docs
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+│ ├── javadoc
+│ │ ├── latest
+│ │ ├── 1.3.1
+│ │ ├── 1.3.0
+│ │ └── ...
+```
+
+## Install
+
+```
+python -m venv mkdocs_env
+source mkdocs_env/bin/activate
+
+pip install -r requirements.txt
+```
+
+## Add git worktrees
+
+For now, I'm just adding a single version
+
+```
+git worktree add home/docs/1.3.1 docs-1.3.1
+git worktree add home/javadoc javadoc
+```
+
+## Build
+
+```
+mkdocs build
+mkdocs serve
+```
+
+## Validate Links
+
+Due to the delayed aggregation of subdocs of `mkdocs-monorepo-plugin` there are some warnings that display for subdocs that mkdocs are not able to connect due to being off by a single directory when the version directory is added, which doesn't mirror the directory layout on disk.
+
+To ensure the links work, I am temporarily using a tool called linkchedker, which will traverse the links on the livesite.
+```
+pip install linkchecker
+
+./linkchecker http://localhost:8000 -r1 -Fcsv/link_warnings.csv
Review Comment:
Do we also need to `cat csv/link_warnings.csv`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341793057
##########
docs-new/home/roadmap.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Roadmap"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Roadmap Overview
+
+This roadmap outlines projects that the Iceberg community is working on, their priority, and a rough size estimate.
+This is based on the latest [community priority discussion](https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E).
+Each high-level item links to a Github project board that tracks the current status.
+Related design docs will be linked on the planning boards.
+
+# Priority 1
+
+* API: [Iceberg 1.0.0](https://github.com/apache/iceberg/projects/3) [medium]
Review Comment:
Looks like this is out of date. Iceberg 1.0 was released a long time ago. Do we have a more recent copy of this roadmap page?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341793057
##########
docs-new/home/roadmap.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Roadmap"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Roadmap Overview
+
+This roadmap outlines projects that the Iceberg community is working on, their priority, and a rough size estimate.
+This is based on the latest [community priority discussion](https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E).
+Each high-level item links to a Github project board that tracks the current status.
+Related design docs will be linked on the planning boards.
+
+# Priority 1
+
+* API: [Iceberg 1.0.0](https://github.com/apache/iceberg/projects/3) [medium]
Review Comment:
Looks like this is out of date. Iceberg 1.0 was released a long time ago. Do we have a more recent copy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343924855
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
Got it, does it make sense to just grab the highest semvar number after the build and copy that directory and rename it to `latest`. Then update what I currently call `latest` to `dev`?
I like `dev` as it basically implies whenever the last build was. We could even consider adding the shortened commit hash to the label `dev (<hash>)` WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341818365
##########
docs-new/LICENSE:
##########
@@ -0,0 +1,201 @@
+ Apache License
Review Comment:
Yup I'll remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1774192281
I'm going to merge this to unblock the next phase of work. Building the site works and I think once this is in we can move on to:
1. Update the style to match the current site
2. Add release branches for the older docs
3. Add automation scripts to build the site
I'd also like to move the docs in `main` from `nightly` to a `docs` area (its own mkdocs site) and remove duplicate markdown files. I think that's probably the next step.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] Fokko commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1338140111
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
Review Comment:
I think some cache id is missing here. I would suggest leaving out the caching initially.
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
Review Comment:
I think this is an issue:
> Adjust the structure to match the layout of the website itself, where ./home/*.md contains the evergreen content and ./home/docs/<version>/*.md contains the versions of Iceberg.
This would mean that it shows features that haven't been released yet. I think it is great to build these pages, but we have to clearly state that it is the dev version. I think Arrow is a nice example:
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
In a follow up PR I think it is nice to add the `strict` flag to avoid slipping in errors.
```suggestion
- run: mkdocs gh-deploy --force --strict
```
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
This will overwrite the PyIceberg site 🕺
I can prioritize the migration to a separate repo. I think we should get these in and then we're good to go: https://github.com/apache/iceberg/pull/8622 https://github.com/apache/iceberg/pull/8652 https://github.com/apache/iceberg/pull/8651
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
This will overwrite the PyIceberg site 🕺
I can prioritize the migration to a separate repo. I think we should get these in and then we're good to go: https://github.com/apache/iceberg/pull/8622 https://github.com/apache/iceberg/pull/8652 https://github.com/apache/iceberg/pull/8651
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
Just a heads-up for PyIceberg I had to jump through some hoops because it also requires a `CNAME` file: https://github.com/apache/iceberg/blob/master/.github/workflows/python-ci-docs.yml#L46-L56 Probably we also have to make sure that we deprecate the asf-site branch that currently hosts the docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341810289
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
Review Comment:
Branches seem to be the more clear choice for both old versions and javadoc. Is that not possible for some reason?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341936141
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
Review Comment:
Yeah, but in mkdocs `site` is reserved for the generated site for each mkdocs project. We can also overwrite this, but I wanted to follow convention as much as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: WIP: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1340866660
##########
docs-new/.github/workflows/ci.yml:
##########
@@ -0,0 +1,29 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - run: |
+ git fetch --tags
+ git worktree add home/docs/1.3.1 docs-1.3.1
+ git worktree add home/javadoc javadoc
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install -r requirements.txt
+ - run: mkdocs gh-deploy --force
Review Comment:
Ah got it, [I've ripped it out temporarily](https://github.com/apache/iceberg/blob/000344ebe85e26e8a97475494adbb5e62103d929/docs-new/.gitignore#L2) for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341859894
##########
docs-new/home/ASF.md:
##########
@@ -0,0 +1,18 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+Apache Software Foundation links
Review Comment:
I need to play with this again, mkdocs wasn't liking that I had an entire section without any markdown file and only external links so I added this as an index, I imagine there's some trick to this to not do what I'm doing now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341781190
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
Review Comment:
What is this tree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341778841
##########
docs-new/home/assets/images/contact-bg.jpg:
##########
Review Comment:
I don't think we need this. It was part of the theme of the Hugo site that we replaced with the Penguin image.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] bitsondatadev commented on a diff in pull request #8659: Phase 1 - New Docs Deployment
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1341866343
##########
docs-new/home/roadmap.md:
##########
@@ -0,0 +1,47 @@
+---
+title: "Roadmap"
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# Roadmap Overview
+
+This roadmap outlines projects that the Iceberg community is working on, their priority, and a rough size estimate.
+This is based on the latest [community priority discussion](https://lists.apache.org/thread.html/r84e80216c259c81f824c6971504c321cd8c785774c489d52d4fc123f%40%3Cdev.iceberg.apache.org%3E).
+Each high-level item links to a Github project board that tracks the current status.
+Related design docs will be linked on the planning boards.
+
+# Priority 1
+
+* API: [Iceberg 1.0.0](https://github.com/apache/iceberg/projects/3) [medium]
Review Comment:
We haven't merged it yet. I need to get back to that one. Perhaps we can get it added to 1.4.0? Otherwise I'd like to avoid changing content yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi commented on PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#issuecomment-1747740766
My primary concern of moving the docs into the main repo was versioning and pollution. It seems like `git worktree` should solve that. I deployed this locally, it seems pretty straightforward.
Do we need to adjust our `.gitignore`? I did see some extra files when deploying locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "bitsondatadev (via GitHub)" <gi...@apache.org>.
bitsondatadev commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1343890396
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
As an aside, we're already going to need our own version of mkdocs monorepo until we can get the changes to accept '.' character in the url. Should we consider adding an ordering feature to this that would enable asc/desc for semver and ignoring non semver versions in the wildcard? That would make it one less thing we'd need to maintain and a valuable feature for a lot of folks trying to do the same thing we are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348076686
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
Review Comment:
I think `nightly` is a more obvious name that it is unreleased. I think `dev` is pretty generic and may be misinterpreted.
I'd also keep it simple with `latest`. It's better to have `latest` pointing to old docs than to have it break because of some automated change. I'd point it to the right branch specifically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348075495
##########
docs-new/mkdocs.yml:
##########
@@ -0,0 +1,96 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+site_name: Apache Iceberg
+docs_dir: home
+
+theme:
+ name: material
+ language: en
+ logo: assets/images/iceberg-logo-icon.png
+ favicon: assets/images/favicon-96x96.png
+ features:
+ - navigation.tabs
+ - navigation.path
+ - navigation.top
+ - navigation.tracking
+ - toc.follow
+ - search.suggest
+ - search.highlight
+ - content.tabs.link
+ - content.code.copy
+ - content.code.annotate
+
+plugins:
+ - search
+ - macros:
+ include_yaml:
+ - variables.yml
+ - monorepo
+
+nav:
+ - Quickstart:
+ - Hive: hive-quickstart.md
+ - Spark: spark-quickstart.md
+ - Docs:
+ - latest: '!include home/docs/latest/mkdocs.yml'
+ - 1.3.1: '!include home/docs/1.3.1/mkdocs.yml'
Review Comment:
Let's open issues and discuss it with the mkdocs monorepo author. I don't think ordering is something we should be distracted by.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348087991
##########
docs-new/README.md:
##########
@@ -0,0 +1,123 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+# making-iceberg-docs
+
+Testbed for the Iceberg docs.
+
+## Requirements
+
+* Python >=3.9
+* pip
+
+
+## Build process
+
+The directory structure is intended to mimic the tree hierarchy of the website. This will enable contributors to find the documentation they need to update easier. The static and documentation will reside in the same location.
+
+All static pages are all the `./home/.*md` files and the docs are the `.home/docs/<version>/*.md` files. Notice the location of the `mkdocs.yml`.
+
+Looking at this though, you may ask where the older versions and javadocs are.
+
+```
+.
+├── home
+│ ├── assets
+│ │ └── images
+│ ├── docs
+│ │ └── latest
+│ │ ├── assets
+│ │ ├── api.md
+│ │ ├── aws.md
+│ │ ├── branching.md
+│ │ ├── ...
+│ │ ├── **mkdocs.yml(docs)**
+│ │ ├── ...
+│ │ ├── spark-structured-streaming.md
+│ │ ├── spark-writes.md
+│ │ └── table-migration.md
+│ ├── ASF.md
+│ ├── about.md
+│ ├── ...
+│ ├── vendors.md
+│ └── view-spec.md
+├── LICENSE
+├── README.md
+├── **mkdocs.yml(static)**
+├── requirements.txt
+└── variables.yml
+```
+
+All of the documentation versions are saved in special `docs-<version>` tags that only contain the root of the docs version they contian. There is also a `javadoc` tag that contains all prior versions of the javadocs in a single tag. These are generated and loaded only at build time using the [git-worktree](https://git-scm.com/docs/git-worktree) docs.
+
+```
Review Comment:
Can you add some description so other readers know what this is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Phase 1 - New Docs Deployment [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #8659:
URL: https://github.com/apache/iceberg/pull/8659#discussion_r1348087050
##########
docs-new/.github/bin/deploy_docs.sh:
##########
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+while [[ "$#" -gt 0 ]]; do
+ case $1 in
+ -v|--version) ICEBERG_VERSION="$2"; shift ;;
+ *) echo "Unknown parameter passed: $1"; exit 1 ;;
+ esac
+ shift
+done
+
+GIT_BRANCH="docs-${ICEBERG_VERSION}"
+
+# change to branch
+git checkout -b $GIT_BRANCH
+
+#remove all local files and leave site/ dir
+rm ./*
Review Comment:
How does this leave the `site` dir?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org