You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "xuzifu666 (via GitHub)" <gi...@apache.org> on 2025/01/16 06:19:10 UTC
[PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
xuzifu666 opened a new pull request, #4926:
URL: https://github.com/apache/paimon/pull/4926
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #xxx
Currently user delete tags can only according to tag names，actually user may need delete their tags according to a time （such as history auto-create tags which not contain retained time），need introduce a quick way to delete all tags which earlier than a time，this pr is aimed to support it.
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "askwang (via GitHub)" <gi...@apache.org>.
askwang commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594997666
> So my suggestion is introducing `expire_tags` procedure, this procedure should be similiar to `expire_snapshots` procedure.
The expire_tags procedure is already supported, maybe can meet this demand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "xuzifu666 (via GitHub)" <gi...@apache.org>.
xuzifu666 commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594691403
> Is it possible to just add parameters to the original `delelte_tag`?
Thanks, can only add a new params in delete_tag.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "LinMingQiang (via GitHub)" <gi...@apache.org>.
LinMingQiang commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594655307
Is it possible to just add parameters to the original `delelte_tag`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594975719
So my suggestion is introducing expire_tags
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "xuzifu666 (via GitHub)" <gi...@apache.org>.
xuzifu666 commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2595007259
@JingsongLi @askwang master branch had according to file last modified time to expire tags，so this pr can be close first，Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "LinMingQiang (via GitHub)" <gi...@apache.org>.
LinMingQiang commented on code in PR #4926:
URL: https://github.com/apache/paimon/pull/4926#discussion_r1917856404
##########
paimon-flink/paimon-flink-1.18/src/main/java/org/apache/paimon/flink/procedure/DeleteTagsByTimestampProcedure.java:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.flink.procedure;
+
+import org.apache.paimon.catalog.Catalog;
+import org.apache.paimon.catalog.Identifier;
+import org.apache.paimon.fs.FileIO;
+import org.apache.paimon.fs.FileStatus;
+import org.apache.paimon.table.FileStoreTable;
+import org.apache.paimon.table.Table;
+import org.apache.paimon.utils.TagManager;
+
+import org.apache.flink.table.procedure.ProcedureContext;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Delete tag by timestamp procedure. Usage:
+ *
+ * <pre><code>
+ * CALL sys.delete_tags_timestamp('tableId', 1736932554000)
+ * </code></pre>
+ */
+public class DeleteTagsByTimestampProcedure extends ProcedureBase {
+ public static final String IDENTIFIER = "delete_tags_timestamp";
+
+ public String[] call(ProcedureContext procedureContext, String tableId, Long timestamp)
+ throws Catalog.TableNotExistException {
+ Table table = catalog.getTable(Identifier.fromString(tableId));
+ FileStoreTable fileStoreTable = (FileStoreTable) table;
+ TagManager tagManager = fileStoreTable.tagManager();
+ FileIO fileIO = fileStoreTable.fileIO();
+ List<String> deleteTags = new ArrayList<>();
+ try {
+ for (String tagName : fileStoreTable.tagManager().allTagNames()) {
+ FileStatus fileStatus = fileIO.listStatus(tagManager.tagPath(tagName))[0];
+ if (fileStatus.getModificationTime() < timestamp) {
+ deleteTags.add(tagName);
+ }
+ }
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+
+ if (deleteTags.size() > 0) {
+ table.deleteTags(String.join(",", deleteTags));
+ }
+
+ return new String[] {"Success"};
+ }
Review Comment:
Returns the all tagnames that have been deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "askwang (via GitHub)" <gi...@apache.org>.
askwang commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594994564
The existed expire_tags procedure maybe can meet this demand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #4926:
URL: https://github.com/apache/paimon/pull/4926#issuecomment-2594975131
Time related deletion should be defined through the behavior of expiration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp [paimon]
Posted by "xuzifu666 (via GitHub)" <gi...@apache.org>.
xuzifu666 closed pull request #4926: [core] Introduce DeleteTagsByTimestampProcedure to delete tags by timestamp
URL: https://github.com/apache/paimon/pull/4926
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org