You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/26 15:56:57 UTC
[GitHub] [iceberg-docs] samredai opened a new pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
samredai opened a new pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18
This adds support for [Lottie Files](https://lottiefiles.com/) which makes it easy to add high-res and smooth animations. This is integrated into the front-matter for the landing-page content so you can set `LottieFile: a-lottie-animation.json` for one of the splash page blogs and it will include a properly sized animation next to the feature title and description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792885628
##########
File path: landing-page/content/services/data-compaction.html
##########
@@ -1,7 +1,8 @@
---
-Title: Time Travel
-AsciinemaCast: time_travel.cast
-Category: Post
+Title: Data Compaction
+Description: Data compaction is supported out-of-the-box and you can choose from different rewrite strategies such as bin-packing, sorting, or z-ordering to optimize file layout and size.
+LottieFile: data-compaction-animation.json
Review comment:
I like having an animation, but I don't think that this represents data compaction well. It looks like files are being split instead.
What about having another command box that shows the compaction command, `CALL system.rewrite_data_files(...)`? Then we can link directly to the compaction docs (in 0.13.0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r793987792
##########
File path: landing-page/content/services/data-compaction.html
##########
@@ -1,7 +1,8 @@
---
-Title: Time Travel
-AsciinemaCast: time_travel.cast
-Category: Post
+Title: Data Compaction
+Description: Data compaction is supported out-of-the-box and you can choose from different rewrite strategies such as bin-packing, sorting, or z-ordering to optimize file layout and size.
+LottieFile: data-compaction-animation.json
Review comment:
Might want to add a couple options to demonstrate, like `where` to show how to compact a single partition and `strategy` to show bin packing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] samredai commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
samredai commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r793973419
##########
File path: landing-page/content/services/data-compaction.html
##########
@@ -1,7 +1,8 @@
---
-Title: Time Travel
-AsciinemaCast: time_travel.cast
-Category: Post
+Title: Data Compaction
+Description: Data compaction is supported out-of-the-box and you can choose from different rewrite strategies such as bin-packing, sorting, or z-ordering to optimize file layout and size.
Review comment:
Ah ok, removed in [b033a57](https://github.com/apache/iceberg-docs/pull/18/commits/b033a571fdb082c317a5987ed1ec5705d3bdce6c)
##########
File path: landing-page/content/services/data-compaction.html
##########
@@ -1,7 +1,8 @@
---
-Title: Time Travel
-AsciinemaCast: time_travel.cast
-Category: Post
+Title: Data Compaction
+Description: Data compaction is supported out-of-the-box and you can choose from different rewrite strategies such as bin-packing, sorting, or z-ordering to optimize file layout and size.
+LottieFile: data-compaction-animation.json
Review comment:
Updated this to show:
```scala
CALL system.rewrite_data_files("nyc.taxis");
```
relevant commit: [b033a57](https://github.com/apache/iceberg-docs/pull/18/commits/b033a571fdb082c317a5987ed1ec5705d3bdce6c)
##########
File path: landing-page/content/services/row-level-deletes.html
##########
@@ -1,9 +1,11 @@
---
-Title: Schema Evolution
-AsciinemaCast: schema_evolution.cast
+Title: Row-level Deletes
+Description: Iceberg supports the copy-on-write and merge-on-read approaches to handle row-level delete operations.
+LottieFile: row-level-deletes-animation.json
Review comment:
Renamed this to "Expressive SQL" and used the code example below:
```
MERGE INTO prod.nyc.taxis pt
USING (SELECT * FROM staging.nyc.taxis) st
ON pt.id = st.id
WHEN NOT MATCHED THEN INSERT *;
```
relevant commit: [b92aebb](https://github.com/apache/iceberg-docs/pull/18/commits/b92aebb3e4550189467840c9c052e13295afc727)
##########
File path: landing-page/content/services/row-level-deletes.html
##########
@@ -1,9 +1,11 @@
---
-Title: Schema Evolution
-AsciinemaCast: schema_evolution.cast
+Title: Row-level Deletes
+Description: Iceberg supports the copy-on-write and merge-on-read approaches to handle row-level delete operations.
Review comment:
I like it, updated!
##########
File path: landing-page/content/services/hidden-partitioning.html
##########
@@ -1,6 +1,8 @@
---
Title: Hidden Partitioning
-Img: partition-spec-evolution.png
+Description: Iceberg handles the tedious and error-prone task of producing partition values for rows in a table and avoids reading unnecessary partitions automatically. Consumers don’t need to know how the table is partitioned or add extra filters to their queries and the partition layouts can evolve as needed.
Review comment:
Fixed! Relevant commit [fea5ec9](https://github.com/apache/iceberg-docs/pull/18/commits/fea5ec9862883a725a7b7d0e5385281d975b8c15)
##########
File path: landing-page/content/services/schema-evolution.html
##########
@@ -0,0 +1,38 @@
+---
+Title: Full Schema Evolution
+Description: Iceberg avoids unpleasant surprises. Schema evolution just works with no "zombie" columns and no un-deleted data when adding columns with previously existing names.
+LearnMore: /docs/latest/evolution/
+Category: Services
+Draft: false
+weight: 100
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+<div id="termynal" data-termynal data-ty-startDelay="600" data-ty-typeDelay="20" data-ty-lineDelay="500">
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="spark-sql>">ALTER TABLE taxis</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">ALTER COLUMN trip_distance</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="" data-ty-delay="2500">TYPE double;</span>
+ <span data-ty>Time taken: 2.594 seconds</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="spark-sql>">ALTER TABLE taxis</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">ALTER COLUMN trip_distance</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">COMMENT ‘The elapsed trip distance in miles reported by the taximeter.’;</span>
+ <span data-ty>Time taken: 0.464 seconds</span>
Review comment:
You're right, the column reorder should definitely be shown here. I updated the second query to be:
```
ALTER TABLE taxis
ALTER COLUMN trip_distance
AFTER fare;
```
relevant commit: [046de22](https://github.com/apache/iceberg-docs/pull/18/commits/046de2214974045f999b2d7f55527fe9c5aa1bfd)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#issuecomment-1023627694
Thanks, @samredai! Merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#issuecomment-1022444819
Is it possible to reorder some of these as well? I think the flexible SQL plans should be near the top. I'd probably go with:
* Flexible SQL updates
* Full schema evolution
* Hidden partitioning
* Time travel and rollback
* Easy maintenance and compaction (we're highlighting the broader set of functionality here)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792886159
##########
File path: landing-page/content/services/row-level-deletes.html
##########
@@ -1,9 +1,11 @@
---
-Title: Schema Evolution
-AsciinemaCast: schema_evolution.cast
+Title: Row-level Deletes
+Description: Iceberg supports the copy-on-write and merge-on-read approaches to handle row-level delete operations.
+LottieFile: row-level-deletes-animation.json
Review comment:
For this one as well, I'm not sure how well the animation represents row-level deletes. I would much rather highlight the MERGE INTO syntax in a SQL box. That's a really useful thing to see!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue merged pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792895531
##########
File path: landing-page/content/services/schema-evolution.html
##########
@@ -0,0 +1,38 @@
+---
+Title: Full Schema Evolution
+Description: Iceberg avoids unpleasant surprises. Schema evolution just works with no "zombie" columns and no un-deleted data when adding columns with previously existing names.
+LearnMore: /docs/latest/evolution/
+Category: Services
+Draft: false
+weight: 100
+---
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements. See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License. You may obtain a copy of the License at
+ -
+ - http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+<div id="termynal" data-termynal data-ty-startDelay="600" data-ty-typeDelay="20" data-ty-lineDelay="500">
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="spark-sql>">ALTER TABLE taxis</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">ALTER COLUMN trip_distance</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="" data-ty-delay="2500">TYPE double;</span>
+ <span data-ty>Time taken: 2.594 seconds</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="spark-sql>">ALTER TABLE taxis</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">ALTER COLUMN trip_distance</span>
+ <span data-ty="input" data-ty-cursor="▋" data-ty-prompt="">COMMENT ‘The elapsed trip distance in miles reported by the taximeter.’;</span>
+ <span data-ty>Time taken: 0.464 seconds</span>
Review comment:
Do you think that column reorder is attractive, or is a comment the better option?
Also, can you make the comment smaller so it fits on one line? You can probably get rid of "the" and "reported by the taximeter."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] samredai commented on pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
samredai commented on pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#issuecomment-1023612515
> Is it possible to reorder some of these as well?
Yep, hugo makes it super simple by just setting the weights in the front-matter. Is it ok if I immediately follow-up with a PR setting the weights?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792891685
##########
File path: landing-page/content/services/row-level-deletes.html
##########
@@ -1,9 +1,11 @@
---
-Title: Schema Evolution
-AsciinemaCast: schema_evolution.cast
+Title: Row-level Deletes
+Description: Iceberg supports the copy-on-write and merge-on-read approaches to handle row-level delete operations.
Review comment:
The term "row-level deletes" probably doesn't connect with people coming to this page. Instead, I'd highlight "Expressive SQL changes" or something similar. What users care about is that they can use `MERGE INTO`, `UPDATE`, and `DELETE FROM`. (Mostly `MERGE` and `DELETE`)
So the description can be something like `Iceberg supports flexible SQL commands to merge new data, update existing rows, and perform targeted deletes. Iceberg can eagerly rewrite data files for read performance, or it can use delete deltas for faster updates.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792894391
##########
File path: landing-page/content/services/hidden-partitioning.html
##########
@@ -1,6 +1,8 @@
---
Title: Hidden Partitioning
-Img: partition-spec-evolution.png
+Description: Iceberg handles the tedious and error-prone task of producing partition values for rows in a table and avoids reading unnecessary partitions automatically. Consumers don’t need to know how the table is partitioned or add extra filters to their queries and the partition layouts can evolve as needed.
Review comment:
I'd remove "consumers" because I don't think it is clear who a "consumer" is. Instead, I'd use a more direct phrase, like "No extra filters are needed for fast queries, and table layout can be updated as data or queries change."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792892723
##########
File path: landing-page/content/services/hidden-partitioning.html
##########
@@ -1,6 +1,8 @@
---
Title: Hidden Partitioning
-Img: partition-spec-evolution.png
+Description: Iceberg handles the tedious and error-prone task of producing partition values for rows in a table and avoids reading unnecessary partitions automatically. Consumers don’t need to know how the table is partitioned or add extra filters to their queries and the partition layouts can evolve as needed.
Review comment:
"Avoids reading" can be replaced with "skips unnecessary partitions and files automatically"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] rdblue commented on a change in pull request #18: Adding Lottie and integrating lottie file reference into content front-matter
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #18:
URL: https://github.com/apache/iceberg-docs/pull/18#discussion_r792884380
##########
File path: landing-page/content/services/data-compaction.html
##########
@@ -1,7 +1,8 @@
---
-Title: Time Travel
-AsciinemaCast: time_travel.cast
-Category: Post
+Title: Data Compaction
+Description: Data compaction is supported out-of-the-box and you can choose from different rewrite strategies such as bin-packing, sorting, or z-ordering to optimize file layout and size.
Review comment:
Z-ordering is not supported yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org