You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "xicm (via GitHub)" <gi...@apache.org> on 2024/04/29 06:53:11 UTC
[PR] [HUDI-7685] Fix bug in partition TTL [hudi]
xicm opened a new pull request, #11117:
URL: https://github.com/apache/hudi/pull/11117
### Change Logs
After cherry pick https://github.com/apache/hudi/pull/9723, I find the delete partition instant can‘t be committed.
Auto commit in HoodieSparkSqlWriter is disabled, while SparkDeletePartitionCommitActionExecutor calls commitOnAutoCommit to commit the instant.
### Impact
none
### Risk level (write none, low medium or high below)
none
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082298487
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
}, {
"hash" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548",
"triggerID" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 89d11f5e6d030e074350fc70c33f331c98f64d43 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582831482
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
Review Comment:
We can move the initialization of `emptyResult` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582611539
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
+ }
Review Comment:
To avoid writing too many empty replace.commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1584024681
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/exception/HoodieDeletePartitionPendingTableServiceException.java:
##########
@@ -0,0 +1,28 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one
+ * * or more contributor license agreements. See the NOTICE file
+ * * distributed with this work for additional information
+ * * regarding copyright ownership. The ASF licenses this file
+ * * to you under the Apache License, Version 2.0 (the
+ * * "License"); you may not use this file except in compliance
+ * * with the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.hudi.exception;
+
+public class HoodieDeletePartitionPendingTableServiceException extends HoodieDeletePartitionException {
+
Review Comment:
This is not a fatal exception, I want to catch this exception in SparkPartitionTTLActionExecutor instead of throw.
if we just throw this exception, it may affect the ingestion or other table service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082177866
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
}, {
"hash" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548",
"triggerID" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b1358e02b1637a424252a8541c369a3f1b4902c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547)
* 89d11f5e6d030e074350fc70c33f331c98f64d43 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082079879
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b1358e02b1637a424252a8541c369a3f1b4902c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082012156
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b1358e02b1637a424252a8541c369a3f1b4902c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix delete partition instant commit in partition TTL [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #11117:
URL: https://github.com/apache/hudi/pull/11117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1584026036
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
Review Comment:
If HoodieDeletePartitionPendingTableServiceException is caught, return the emptyResult too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582833605
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/exception/HoodieDeletePartitionPendingTableServiceException.java:
##########
@@ -0,0 +1,28 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one
+ * * or more contributor license agreements. See the NOTICE file
+ * * distributed with this work for additional information
+ * * regarding copyright ownership. The ASF licenses this file
+ * * to you under the Apache License, Version 2.0 (the
+ * * "License"); you may not use this file except in compliance
+ * * with the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.hudi.exception;
+
+public class HoodieDeletePartitionPendingTableServiceException extends HoodieDeletePartitionException {
+
Review Comment:
There is no need to add new checked exception, just make the exception msg more clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582832844
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
+ }
LOG.info("Partition ttl find the following expired partitions to delete: " + String.join(",", expiredPartitions));
- return new SparkDeletePartitionCommitActionExecutor<>(context, config, table, instantTime, expiredPartitions).execute();
+ // Auto commit is disabled in config, copy config and enable auto commit for SparkDeletePartitionCommitActionExecutor.
+ HoodieWriteConfig autoCommitConfig = HoodieWriteConfig.newBuilder().withProperties(config.getProps()).withAutoCommit(true).build();
+ return new SparkDeletePartitionCommitActionExecutor<>(context, autoCommitConfig, table, instantTime, expiredPartitions).execute();
Review Comment:
`hoodie.auto.commit` is by default true, why it is disabled specifically?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082090759
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
}, {
"hash" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b1358e02b1637a424252a8541c369a3f1b4902c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547)
* 89d11f5e6d030e074350fc70c33f331c98f64d43 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582611539
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
+ }
Review Comment:
To avoid write too many empty replace.commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1584042433
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/SparkPartitionTTLActionExecutor.java:
##########
@@ -47,11 +49,22 @@ public SparkPartitionTTLActionExecutor(HoodieEngineContext context, HoodieWriteC
@Override
public HoodieWriteMetadata<HoodieData<WriteStatus>> execute() {
+ HoodieWriteMetadata<HoodieData<WriteStatus>> emptyResult = new HoodieWriteMetadata<>();
+ emptyResult.setPartitionToReplaceFileIds(Collections.emptyMap());
+ emptyResult.setWriteStatuses(context.emptyHoodieData());
try {
PartitionTTLStrategy strategy = HoodiePartitionTTLStrategyFactory.createStrategy(table, config.getProps(), instantTime);
List<String> expiredPartitions = strategy.getExpiredPartitionPaths();
+ if (expiredPartitions.isEmpty()) {
+ return emptyResult;
+ }
LOG.info("Partition ttl find the following expired partitions to delete: " + String.join(",", expiredPartitions));
- return new SparkDeletePartitionCommitActionExecutor<>(context, config, table, instantTime, expiredPartitions).execute();
+ // Auto commit is disabled in config, copy config and enable auto commit for SparkDeletePartitionCommitActionExecutor.
+ HoodieWriteConfig autoCommitConfig = HoodieWriteConfig.newBuilder().withProperties(config.getProps()).withAutoCommit(true).build();
+ return new SparkDeletePartitionCommitActionExecutor<>(context, autoCommitConfig, table, instantTime, expiredPartitions).execute();
Review Comment:
Flink side: FlinkWriteClients -> getHoodieClientConfig
Spark side: DataSourceUtils -> createHoodieConfig
all set auto commit to false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1584024681
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/exception/HoodieDeletePartitionPendingTableServiceException.java:
##########
@@ -0,0 +1,28 @@
+/*
+ *
+ * * Licensed to the Apache Software Foundation (ASF) under one
+ * * or more contributor license agreements. See the NOTICE file
+ * * distributed with this work for additional information
+ * * regarding copyright ownership. The ASF licenses this file
+ * * to you under the Apache License, Version 2.0 (the
+ * * "License"); you may not use this file except in compliance
+ * * with the License. You may obtain a copy of the License at
+ * *
+ * * http://www.apache.org/licenses/LICENSE-2.0
+ * *
+ * * Unless required by applicable law or agreed to in writing, software
+ * * distributed under the License is distributed on an "AS IS" BASIS,
+ * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * * See the License for the specific language governing permissions and
+ * * limitations under the License.
+ *
+ */
+
+package org.apache.hudi.exception;
+
+public class HoodieDeletePartitionPendingTableServiceException extends HoodieDeletePartitionException {
+
Review Comment:
This is not a fatal exception, I want to catch this exception in SparkPartitionTTLActionExecutor instead of throw.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11117:
URL: https://github.com/apache/hudi/pull/11117#issuecomment-2082103016
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547",
"triggerID" : "1b1358e02b1637a424252a8541c369a3f1b4902c",
"triggerType" : "PUSH"
}, {
"hash" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548",
"triggerID" : "89d11f5e6d030e074350fc70c33f331c98f64d43",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1b1358e02b1637a424252a8541c369a3f1b4902c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23547)
* 89d11f5e6d030e074350fc70c33f331c98f64d43 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23548)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7685] Fix bug in partition TTL [hudi]
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on code in PR #11117:
URL: https://github.com/apache/hudi/pull/11117#discussion_r1582610908
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/ttl/strategy/KeepByTimeStrategy.java:
##########
@@ -103,7 +103,7 @@ private Map<String, Option<String>> getLastCommitTimeForPartitions(List<String>
* @param referenceTime last commit time or creation time for partition
*/
protected boolean isPartitionExpired(String referenceTime) {
- String expiredTime = instantTimePlusMillis(fixInstantTimeCompatibility(referenceTime), ttlInMilis);
+ String expiredTime = instantTimePlusMillis(referenceTime, ttlInMilis);
Review Comment:
fixInstantTimeCompatibility is called in instantTimePlusMillis
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org