You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "zhuangchong (via GitHub)" <gi...@apache.org> on 2023/07/18 05:38:10 UTC
[GitHub] [incubator-paimon] zhuangchong opened a new pull request, #1586: [action] Support Reset next snapshot id of consumer id.
zhuangchong opened a new pull request, #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #1574
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
FlinkActionsE2eTest#testRecordConsumer
ConsumerActionITCase#testRecordConsumer
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1271932582
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ResetConsumerAction.java:
##########
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.paimon.consumer.Consumer;
+import org.apache.paimon.consumer.ConsumerManager;
+import org.apache.paimon.table.FileStoreTable;
+
+import java.util.Map;
+
+/** Reset consumer action for Flink. */
+public class ResetConsumerAction extends TableActionBase {
+
+ private final String consumerId;
+ private final long nextSnapshotId;
+
+ protected ResetConsumerAction(
+ String warehouse,
+ String databaseName,
+ String tableName,
+ Map<String, String> catalogConfig,
+ String consumerId,
+ long nextSnapshotId) {
+ super(warehouse, databaseName, tableName, catalogConfig);
+ this.consumerId = consumerId;
+ this.nextSnapshotId = nextSnapshotId;
+ }
+
+ @Override
+ public void run() throws Exception {
+ FileStoreTable dataTable = (FileStoreTable) table;
+ ConsumerManager consumerManager =
+ new ConsumerManager(dataTable.fileIO(), dataTable.location());
+ consumerManager.recordConsumer(consumerId, new Consumer(nextSnapshotId));
Review Comment:
resetConsumer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] yuzelin commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "yuzelin (via GitHub)" <gi...@apache.org>.
yuzelin commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1270312555
##########
docs/content/maintenance/manage-consumers.md:
##########
@@ -0,0 +1,73 @@
+---
+title: "Manage Consumers"
+weight: 9
+type: docs
+aliases:
+- /maintenance/manage-consumers.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Consumers
+
+## Record Consumer
+
+You can add or reset a consumer with a given consumer ID and next snapshot ID.
+
+{{< hint info >}}
+First, you need to stop the streaming task using this consumer ID, and then execute the record consumer action job.
+{{< /hint >}}
+
+{{< tabs "record-consumer" >}}
+
+{{< tab "Flink" >}}
+
+ Run the following command:
+
+```bash
+<FLINK_HOME>/bin/flink run \
+ /path/to/paimon-flink-action-{{< version >}}.jar \
+ record-consumer \
+ --warehouse <warehouse-path> \
+ --database <database-name> \
+ --table <table-name> \
+ --consumer-id <consumer-id> \
+ --snapshot <next-snapshot-id> \
Review Comment:
How about renaming this argument to `--next-snapshot`?
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/consumer/RecordConsumerAction.java:
##########
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action.consumer;
+
+import org.apache.paimon.flink.action.TableActionBase;
+
+import java.util.Map;
+
+/** Record consumer action for Flink. */
+public class RecordConsumerAction extends TableActionBase {
Review Comment:
I think it's unnecessary to place this action to a new package folder.
##########
paimon-e2e-tests/src/test/java/org/apache/paimon/tests/FlinkActionsE2eTest.java:
##########
@@ -400,6 +400,79 @@ public void testCreateAndDeleteTag() throws Exception {
checkResult();
}
+ @Test
+ public void testRecordConsumer() throws Exception {
+ String tableDdl =
+ "CREATE TABLE IF NOT EXISTS T (\n"
+ + " k INT,\n"
+ + " v STRING,\n"
+ + " PRIMARY KEY (k) NOT ENFORCED\n"
+ + ");\n";
+
+ String inserts =
+ "INSERT INTO T VALUES (1, 'Apache');\n"
+ + "INSERT INTO T VALUES (2, 'Flink');\n"
+ + "INSERT INTO T VALUES (3, 'Paimon');\n";
+
+ runSql("SET 'table.dml-sync' = 'true';\n" + inserts, catalogDdl, useCatalogCmd, tableDdl);
+
+ runSql(
+ "SET 'execution.checkpointing.interval' = '5s';\n"
+ + "INSERT INTO result1 SELECT * FROM T /*+ OPTIONS('consumer-id'='myid') */;",
+ catalogDdl,
+ useCatalogCmd,
+ tableDdl,
+ createResultSink("result1", "k INT, v STRING"));
+ checkResult("1, Apache", "2, Flink", "3, Paimon");
+ clearCurrentResults();
+
+ runSql(
+ "SET 'execution.runtime-mode' = 'batch';\n"
+ + "RESET 'execution.checkpointing.interval';\n"
+ + "INSERT INTO _consumers1 SELECT consumer_id, next_snapshot_id FROM T\\$consumers;",
+ catalogDdl,
+ useCatalogCmd,
+ tableDdl,
+ createResultSink("_consumers1", "consumer_id STRING, next_snapshot_id BIGINT"));
+ checkResult("myid, 4");
+ clearCurrentResults();
+
+ // run reset consumer action job
+ Container.ExecResult execResult =
+ jobManager.execInContainer(
+ "bin/flink",
+ "run",
+ "-p",
+ "1",
+ "lib/paimon-flink-action.jar",
+ "record-consumer",
+ "--warehouse",
+ warehousePath,
+ "--database",
+ "default",
+ "--table",
+ "T",
+ "--consumer-id",
+ "myid",
+ "--snapshot",
+ "1");
+
+ LOG.info(execResult.getStdout());
+ LOG.info(execResult.getStderr());
+
+ runSql(
+ "INSERT INTO _consumers2 SELECT consumer_id, next_snapshot_id FROM T\\$consumers;",
+ catalogDdl,
+ useCatalogCmd,
+ tableDdl,
+ createResultSink("_consumers2", "consumer_id STRING, next_snapshot_id BIGINT"));
+
+ // The value of next_snapshot_id in the result is 4 instead of 1,
+ // because the streaming read task is running, and the consumer `myid` is reset again,
+ // overwriting the value written by the action job.
+ checkResult("myid, 4");
+ }
Review Comment:
I think we can remove this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#issuecomment-1645265127
> Thanks for contributing~ left some comments.
@yuzelin Thanks. done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi merged pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1271932779
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ResetConsumerActionFactory.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Factory to create {@link ResetConsumerAction}. */
+public class ResetConsumerActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "reset-consumer";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ checkRequiredArgument(params, "consumer-id");
+ checkRequiredArgument(params, "next-snapshot");
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+ String consumerId = params.get("consumer-id");
+ long nextSnapshotId = Long.parseLong(params.get("next-snapshot"));
+
+ ResetConsumerAction action =
+ new ResetConsumerAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ catalogConfig,
+ consumerId,
+ nextSnapshotId);
+ return Optional.of(action);
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"reset-consumer\" record a consumer from the given next snapshot.");
+ System.out.println();
+
+ System.out.println("Syntax:");
+ System.out.println(
+ " record-consumer --warehouse <warehouse-path> --database <database-name> "
Review Comment:
reset
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/ResetConsumerActionFactory.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.action;
+
+import org.apache.flink.api.java.tuple.Tuple3;
+import org.apache.flink.api.java.utils.MultipleParameterTool;
+
+import java.util.Map;
+import java.util.Optional;
+
+/** Factory to create {@link ResetConsumerAction}. */
+public class ResetConsumerActionFactory implements ActionFactory {
+
+ public static final String IDENTIFIER = "reset-consumer";
+
+ @Override
+ public String identifier() {
+ return IDENTIFIER;
+ }
+
+ @Override
+ public Optional<Action> create(MultipleParameterTool params) {
+ checkRequiredArgument(params, "consumer-id");
+ checkRequiredArgument(params, "next-snapshot");
+
+ Tuple3<String, String, String> tablePath = getTablePath(params);
+ Map<String, String> catalogConfig = optionalConfigMap(params, "catalog-conf");
+ String consumerId = params.get("consumer-id");
+ long nextSnapshotId = Long.parseLong(params.get("next-snapshot"));
+
+ ResetConsumerAction action =
+ new ResetConsumerAction(
+ tablePath.f0,
+ tablePath.f1,
+ tablePath.f2,
+ catalogConfig,
+ consumerId,
+ nextSnapshotId);
+ return Optional.of(action);
+ }
+
+ @Override
+ public void printHelp() {
+ System.out.println(
+ "Action \"reset-consumer\" record a consumer from the given next snapshot.");
Review Comment:
reset
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1271839297
##########
docs/content/maintenance/manage-consumers.md:
##########
@@ -0,0 +1,73 @@
+---
+title: "Manage Consumers"
+weight: 9
+type: docs
+aliases:
+- /maintenance/manage-consumers.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Consumers
+
+## Record Consumer
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] yuzelin commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "yuzelin (via GitHub)" <gi...@apache.org>.
yuzelin commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1270486827
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/TableActionBase.java:
##########
@@ -57,7 +57,7 @@ public abstract class TableActionBase extends ActionBase {
protected Table table;
- TableActionBase(
+ protected TableActionBase(
Review Comment:
This can be reset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1271770379
##########
paimon-core/src/main/java/org/apache/paimon/table/Table.java:
##########
@@ -79,6 +79,9 @@ public interface Table extends Serializable {
@Experimental
void rollbackTo(String tagName);
+ /** Record the consumer of the table through consumer id and next snapshot id. */
+ void recordConsumer(String consumerId, long nextSnapshotId);
Review Comment:
Don't introduce public API now. It is not so stable.
##########
docs/content/maintenance/manage-consumers.md:
##########
@@ -0,0 +1,73 @@
+---
+title: "Manage Consumers"
+weight: 9
+type: docs
+aliases:
+- /maintenance/manage-consumers.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Consumers
+
+## Record Consumer
+
+You can add or reset a consumer with a given consumer ID and next snapshot ID.
+
+{{< hint info >}}
+First, you need to stop the streaming task using this consumer ID, and then execute the record consumer action job.
+{{< /hint >}}
+
+{{< tabs "record-consumer" >}}
Review Comment:
Is there a better name?
##########
docs/content/maintenance/manage-consumers.md:
##########
@@ -0,0 +1,73 @@
+---
+title: "Manage Consumers"
+weight: 9
+type: docs
+aliases:
+- /maintenance/manage-consumers.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Consumers
+
+## Record Consumer
Review Comment:
Just document this with consumer. in query-table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1270528305
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/TableActionBase.java:
##########
@@ -57,7 +57,7 @@ public abstract class TableActionBase extends ActionBase {
protected Table table;
- TableActionBase(
+ protected TableActionBase(
Review Comment:
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] yuzelin commented on pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "yuzelin (via GitHub)" <gi...@apache.org>.
yuzelin commented on PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#issuecomment-1645434630
Not sure that whether it's necessary to introduce an method in public `Table` interface.
@JingsongLi Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#issuecomment-1647616066
> +1
Thanks. @JingsongLi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] zhuangchong commented on a diff in pull request #1586: [action] Support Reset next snapshot id of consumer id.
Posted by "zhuangchong (via GitHub)" <gi...@apache.org>.
zhuangchong commented on code in PR #1586:
URL: https://github.com/apache/incubator-paimon/pull/1586#discussion_r1271839885
##########
docs/content/maintenance/manage-consumers.md:
##########
@@ -0,0 +1,73 @@
+---
+title: "Manage Consumers"
+weight: 9
+type: docs
+aliases:
+- /maintenance/manage-consumers.html
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Manage Consumers
+
+## Record Consumer
+
+You can add or reset a consumer with a given consumer ID and next snapshot ID.
+
+{{< hint info >}}
+First, you need to stop the streaming task using this consumer ID, and then execute the record consumer action job.
+{{< /hint >}}
+
+{{< tabs "record-consumer" >}}
Review Comment:
Has been modified to `reset-consumer`.
##########
paimon-core/src/main/java/org/apache/paimon/table/Table.java:
##########
@@ -79,6 +79,9 @@ public interface Table extends Serializable {
@Experimental
void rollbackTo(String tagName);
+ /** Record the consumer of the table through consumer id and next snapshot id. */
+ void recordConsumer(String consumerId, long nextSnapshotId);
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org