You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "leaves12138 (via GitHub)" <gi...@apache.org> on 2023/07/21 08:46:46 UTC
[GitHub] [incubator-paimon] leaves12138 opened a new pull request, #1624: [improve] support zorder rewrite the unaware-bucket table
leaves12138 opened a new pull request, #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624
This class are copied from iceberg:
paimon-core/src/main/java/org/apache/paimon/utils/ByteBuffers.java
paimon-core/src/main/java/org/apache/paimon/utils/ZOrderByteUtils.java
paimon-core/src/test/java/org/apache/paimon/utils/TestZOrderByteUtil.java
This bytes(every column devoted to z-index) is fixed to 8 bytes, so if the column is short or byte etc, it will fill the upper byte with zero. If the column is string or something like byte[] more then 8, it will truncate it to 8 byte.
The action will invoke like this:
Table(sql select) ----------------> DataStream[Row] -------------------> DataStream[Row] -----------------> Table(with z-order) -----------------> Table(sorted) -----[overwrite target]-----> Table(overwrited)
usage example:
<flink run paimon jar>
zorder-rewrite
--warehouse
/Users/yejunhao/paimontest/GlobalBatchReadWrite
--database
my_db
--table
Orders1
--sql-select
"SELECT * FROM my_db.Orders1 WHERE f0 = 0"
--zorder-by
f0,f1,f2,f3,f4,f7,f8,f9,f10,f11,f12,f13,f14,f15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] YannByron commented on pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "YannByron (via GitHub)" <gi...@apache.org>.
YannByron commented on PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#issuecomment-1664943934
I notice that most of codes is in Flink. Can extract more codes to the common/core module so that other engines (like spark) can use them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1277226221
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/FlinkActionEnvironmentBase.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.paimon.catalog.Identifier;
+import org.apache.paimon.flink.LogicalTypeConversion;
+import org.apache.paimon.types.DataType;
+import org.apache.paimon.types.DataTypeCasts;
+
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.table.api.EnvironmentSettings;
+import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
+import org.apache.flink.table.types.logical.LogicalType;
+
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/** Flink Action Base with batch environment and stream environment. */
+public abstract class FlinkActionEnvironmentBase extends ActionBase {
Review Comment:
You can just put this into `ActionBase`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] leaves12138 commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1277331622
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/OrderRewriteActionFactory.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Action Factory to create {@link OrderRewriteAction}. */
+public class OrderRewriteActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "order-rewrite";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ if (params.has("help")) {
+ printHelp();
+ return Optional.empty();
+ }
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+
+ String sqlSelect = getSqlSelect(params);
+
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+
+ String sqlOrderBy = getSqlOrderBy(params);
+
+ return Optional.of(
+ new OrderRewriteAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ sqlSelect,
+ sqlOrderBy,
+ catalogConfig));
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"order-rewrite\" is similar to sql \"INSERT OVERWRITE target_table SELECT * FROM SOURCE ZORDER BY col1,col2,... \".");
+ System.out.println();
+
+ System.out.println("Syntax:");
+ System.out.println(
+ " order-rewrite --warehouse <warehouse-path>\n"
+ + " --database <database-name>\n"
+ + " --warehouse <warehouse-path>\n"
+ + " --table <target-table-name>\n"
+ + " --sql-select \"sql\"\n"
Review Comment:
Learned from iceberg: https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_data_files
Iceberg rewrite action has
sort_order: For Zorder use a comma separated list of columns within zorder(). (Supported in Spark 3.2 and Above) Example: zorder(c1,c2,c3). Else, Comma separated sort orders in the format (ColumnName SortDirection NullOrder). Where SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST. Defaults to the table’s sort order
where:predicate as a string used for filtering the files. Note that all files that may contain data matching the filter will be selected for rewriting
eg:
CALL catalog_name.system.rewrite_data_files(table => 'db.sample', where => 'id = 3 and name = "foo", strategy => 'sort', sort_order => 'zorder(c1,c2)')
But I thing use write sql-select is more flexible, which mean they could rewrite source table to target table by zorder without changing origin table. So i change "where" to "sql-select"
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/OrderRewriteActionFactory.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Action Factory to create {@link OrderRewriteAction}. */
+public class OrderRewriteActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "order-rewrite";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ if (params.has("help")) {
+ printHelp();
+ return Optional.empty();
+ }
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+
+ String sqlSelect = getSqlSelect(params);
+
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+
+ String sqlOrderBy = getSqlOrderBy(params);
+
+ return Optional.of(
+ new OrderRewriteAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ sqlSelect,
+ sqlOrderBy,
+ catalogConfig));
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"order-rewrite\" is similar to sql \"INSERT OVERWRITE target_table SELECT * FROM SOURCE ZORDER BY col1,col2,... \".");
+ System.out.println();
+
+ System.out.println("Syntax:");
+ System.out.println(
+ " order-rewrite --warehouse <warehouse-path>\n"
+ + " --database <database-name>\n"
+ + " --warehouse <warehouse-path>\n"
+ + " --table <target-table-name>\n"
+ + " --sql-select \"sql\"\n"
Review Comment:
Learned from iceberg: https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_data_files
Iceberg rewrite action has
sort_order: For Zorder use a comma separated list of columns within zorder(). (Supported in Spark 3.2 and Above) Example: zorder(c1,c2,c3). Else, Comma separated sort orders in the format (ColumnName SortDirection NullOrder). Where SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST. Defaults to the table’s sort order
where:predicate as a string used for filtering the files. Note that all files that may contain data matching the filter will be selected for rewriting
eg:
CALL catalog_name.system.rewrite_data_files(table => 'db.sample', where => 'id = 3 and name = "foo", strategy => 'sort', sort_order => 'zorder(c1,c2)')
But I thing use write sql-select is more flexible, which mean they could rewrite source table to target table by zorder without changing origin table. So i change "where" to "sql-select"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1277227339
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/OrderRewriteActionFactory.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Action Factory to create {@link OrderRewriteAction}. */
+public class OrderRewriteActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "order-rewrite";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ if (params.has("help")) {
+ printHelp();
+ return Optional.empty();
+ }
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+
+ String sqlSelect = getSqlSelect(params);
+
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+
+ String sqlOrderBy = getSqlOrderBy(params);
+
+ return Optional.of(
+ new OrderRewriteAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ sqlSelect,
+ sqlOrderBy,
+ catalogConfig));
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"order-rewrite\" is similar to sql \"INSERT OVERWRITE target_table SELECT * FROM SOURCE ZORDER BY col1,col2,... \".");
+ System.out.println();
+
+ System.out.println("Syntax:");
+ System.out.println(
+ " order-rewrite --warehouse <warehouse-path>\n"
+ + " --database <database-name>\n"
+ + " --warehouse <warehouse-path>\n"
+ + " --table <target-table-name>\n"
+ + " --sql-select \"sql\"\n"
Review Comment:
Should we have sql-select and sql-order-by? Can you list some reference system?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] leaves12138 commented on pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#issuecomment-1685503667
This pull request re-open as https://github.com/apache/incubator-paimon/pull/1846
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1277225597
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ActionFactory.java:
##########
@@ -160,4 +160,20 @@ static void parseKeyValueString(Map<String, String> map, String kvString) {
}
map.put(kv[0].trim(), kv[1].trim());
}
+
+ default String getSqlOrderBy(MultipleParameterTool params) {
Review Comment:
Why in base class?
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ActionFactory.java:
##########
@@ -160,4 +160,20 @@ static void parseKeyValueString(Map<String, String> map, String kvString) {
}
map.put(kv[0].trim(), kv[1].trim());
}
+
+ default String getSqlOrderBy(MultipleParameterTool params) {
+ String sqlOrderBy = params.get("sql-order-by");
+ if (sqlOrderBy == null) {
+ throw new IllegalArgumentException("Please specify \"sql-order-by\".");
+ }
+ return sqlOrderBy;
+ }
+
+ default String getSqlSelect(MultipleParameterTool params) {
Review Comment:
Why in base class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] leaves12138 closed pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 closed pull request #1624: [improve] support zorder rewrite the unaware-bucket table
URL: https://github.com/apache/incubator-paimon/pull/1624
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] leaves12138 commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "leaves12138 (via GitHub)" <gi...@apache.org>.
leaves12138 commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1277331622
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/OrderRewriteActionFactory.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Action Factory to create {@link OrderRewriteAction}. */
+public class OrderRewriteActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "order-rewrite";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ if (params.has("help")) {
+ printHelp();
+ return Optional.empty();
+ }
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+
+ String sqlSelect = getSqlSelect(params);
+
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+
+ String sqlOrderBy = getSqlOrderBy(params);
+
+ return Optional.of(
+ new OrderRewriteAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ sqlSelect,
+ sqlOrderBy,
+ catalogConfig));
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"order-rewrite\" is similar to sql \"INSERT OVERWRITE target_table SELECT * FROM SOURCE ZORDER BY col1,col2,... \".");
+ System.out.println();
+
+ System.out.println("Syntax:");
+ System.out.println(
+ " order-rewrite --warehouse <warehouse-path>\n"
+ + " --database <database-name>\n"
+ + " --warehouse <warehouse-path>\n"
+ + " --table <target-table-name>\n"
+ + " --sql-select \"sql\"\n"
Review Comment:
Learned from iceberg: https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_data_files
Iceberg rewrite action has
sort_order: For Zorder use a comma separated list of columns within zorder(). (Supported in Spark 3.2 and Above) Example: zorder(c1,c2,c3). Else, Comma separated sort orders in the format (ColumnName SortDirection NullOrder). Where SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST. Defaults to the table’s sort order
where:predicate as a string used for filtering the files. Note that all files that may contain data matching the filter will be selected for rewriting
eg:
CALL catalog_name.system.rewrite_data_files(table => 'db.sample', where => 'id = 3 and name = "foo", strategy => 'sort', sort_order => 'zorder(c1,c2)')
But I thing use write sql-select is more flexible, which means they could rewrite source table to target table by zorder without changing origin table. So i change "where" to "sql-select"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1624: [improve] support zorder rewrite the unaware-bucket table
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1624:
URL: https://github.com/apache/incubator-paimon/pull/1624#discussion_r1270666889
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ZorderRewriteAction.java:
##########
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.paimon.flink.zorder.ZorderSorter;
+
+import org.apache.flink.table.api.Table;
+
+import java.util.List;
+import java.util.Map;
+
+/**
+ * Rewrite the target table by z-order with specified columns.
+ *
+ * <p>The effect is similar to sql: "INSERT OVERWRITE target_table SELECT * FROM SOURCE ZORDER BY
+ * col1,col2,... "
+ *
+ * <p>Example usage: zorder-rewrite --warehouse /tmp/paimon/warehouse --database my_db --table
+ * Orders1 --sql-select "SELECT * FROM my_db.Orders1 WHERE f0 < 10" --zorder-by
+ * f0,f1,f2,f3,f4,f7,f8,f9,f10,f11,f12,f13,f14,f15
+ */
+public class ZorderRewriteAction extends FlinkActionEnvironmentBase {
Review Comment:
See https://iceberg.apache.org/docs/latest/spark-procedures/#rewrite_data_files
A `sort_order` to declare z-order.
Maybe we can have a sort action. and with a sort-order to declare z-order too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org