You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "xushiyan (via GitHub)" <gi...@apache.org> on 2023/01/24 02:10:17 UTC
[GitHub] [hudi] xushiyan opened a new pull request, #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
xushiyan opened a new pull request, #7740:
URL: https://github.com/apache/hudi/pull/7740
### Change Logs
When metadata table enables a new partition (e.g., colstats), a deltacommit will be created, which will have suffix `000` as the instant time on metadata table's timeline.
### Impact
deltacommit instant timestamp for such operations will have a new format (suffix `000`)
### Risk level
Medium
### Documentation Update
NA
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1401430211
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a0838c73c28f77406b2865da1c0bf37cdd133822 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577)
* aefedbf6024c8de89e82ca227fbb60ac6647b824 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1401435482
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a0838c73c28f77406b2865da1c0bf37cdd133822 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577)
* aefedbf6024c8de89e82ca227fbb60ac6647b824 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #7740:
URL: https://github.com/apache/hudi/pull/7740#discussion_r1085546347
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1040,7 +1043,13 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
// Trigger compaction with suffixes based on the same instant time. This ensures that any future
// delta commits synced over will not have an instant time lesser than the last completed instant on the
// metadata table.
- final String compactionInstantTime = latestDeltaCommitTimeInMetadataTable + METADATA_COMPACTION_TIME_SUFFIX;
+ if (latestDeltaCommitTimeInMetadataTable.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
+ checkState(latestDeltaCommitTimeInMetadataTable.endsWith(METADATA_TABLE_INIT_TIME_SUFFIX),
Review Comment:
Will this affect compaction of existing metadata partition? If so, we may need to recreate the table in upgrade step. Let's make sure adding a new suffix is backwards compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1404497443
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
}, {
"hash" : "62fc57264429a539a6c242e68297733242a08f3c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "62fc57264429a539a6c242e68297733242a08f3c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 405071a2cab10908738a7c54269aff189ac13f47 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610)
* 62fc57264429a539a6c242e68297733242a08f3c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5464] Suffix instant time for metadata partition init deltacommit [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-5083042136
Suffix instant time for MDT partition-init deltacommit; danny: suffix removed generally in 1.x so not needed. Obsolete for the 1.x timeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1402915037
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 405071a2cab10908738a7c54269aff189ac13f47 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1401336483
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a0838c73c28f77406b2865da1c0bf37cdd133822 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1402776125
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aefedbf6024c8de89e82ca227fbb60ac6647b824 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581)
* 405071a2cab10908738a7c54269aff189ac13f47 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #7740:
URL: https://github.com/apache/hudi/pull/7740#discussion_r1085546347
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1040,7 +1043,13 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
// Trigger compaction with suffixes based on the same instant time. This ensures that any future
// delta commits synced over will not have an instant time lesser than the last completed instant on the
// metadata table.
- final String compactionInstantTime = latestDeltaCommitTimeInMetadataTable + METADATA_COMPACTION_TIME_SUFFIX;
+ if (latestDeltaCommitTimeInMetadataTable.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
+ checkState(latestDeltaCommitTimeInMetadataTable.endsWith(METADATA_TABLE_INIT_TIME_SUFFIX),
Review Comment:
Will this affect compaction of existing metadata partition? If so, we may need to recreate the table in upgrade step.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #7740:
URL: https://github.com/apache/hudi/pull/7740#discussion_r1085740528
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1040,7 +1043,13 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
// Trigger compaction with suffixes based on the same instant time. This ensures that any future
// delta commits synced over will not have an instant time lesser than the last completed instant on the
// metadata table.
- final String compactionInstantTime = latestDeltaCommitTimeInMetadataTable + METADATA_COMPACTION_TIME_SUFFIX;
+ if (latestDeltaCommitTimeInMetadataTable.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
+ checkState(latestDeltaCommitTimeInMetadataTable.endsWith(METADATA_TABLE_INIT_TIME_SUFFIX),
Review Comment:
I would prefer to keep it simple. if latest delta commit refers to an instantiation dc, we can ignore triggering compaction only.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1119,7 +1128,8 @@ private void initialCommit(String createInstantTime, List<MetadataPartitionType>
LOG.info("Committing " + partitions.size() + " partitions and " + totalDataFilesCount + " files to metadata");
- commit(createInstantTime, partitionToRecordsMap, false);
+ commit(createInstantTime.equals(SOLO_COMMIT_TIMESTAMP) ? createInstantTime : createInstantTime + METADATA_TABLE_INIT_TIME_SUFFIX,
Review Comment:
can we write a functional test.
start hudi table w/ default. after 2 commits, enable col stats. after 1 commit succeeds, confirm metadata table timeline is as expected. we have a delta commit for every commit in data table. and we have additional dc w/ "000" suffix. we can even check the commit metadata for this dc to ensure it has data only for col_stats partition and not for "files" partition.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1040,7 +1043,13 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
// Trigger compaction with suffixes based on the same instant time. This ensures that any future
// delta commits synced over will not have an instant time lesser than the last completed instant on the
// metadata table.
- final String compactionInstantTime = latestDeltaCommitTimeInMetadataTable + METADATA_COMPACTION_TIME_SUFFIX;
+ if (latestDeltaCommitTimeInMetadataTable.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
+ checkState(latestDeltaCommitTimeInMetadataTable.endsWith(METADATA_TABLE_INIT_TIME_SUFFIX),
Review Comment:
@codope : any existing table should work w/o any issues. but does not hurt to test it out. @xushiyan : can you test out this scenario once to confirm we are good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1404501154
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
}, {
"hash" : "62fc57264429a539a6c242e68297733242a08f3c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14644",
"triggerID" : "62fc57264429a539a6c242e68297733242a08f3c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 405071a2cab10908738a7c54269aff189ac13f47 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610)
* 62fc57264429a539a6c242e68297733242a08f3c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14644)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1404582197
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
}, {
"hash" : "62fc57264429a539a6c242e68297733242a08f3c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14644",
"triggerID" : "62fc57264429a539a6c242e68297733242a08f3c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 62fc57264429a539a6c242e68297733242a08f3c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14644)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5464] Suffix instant time for metadata partition init deltacommit [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-3652843753
I don't think we need this as we remove the suffix generally since 1.x.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5464] Suffix instant time for metadata partition init deltacommit [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous closed pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
URL: https://github.com/apache/hudi/pull/7740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1401573398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aefedbf6024c8de89e82ca227fbb60ac6647b824 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "xushiyan (via GitHub)" <gi...@apache.org>.
xushiyan commented on code in PR #7740:
URL: https://github.com/apache/hudi/pull/7740#discussion_r1085799867
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1040,7 +1043,13 @@ protected void compactIfNecessary(BaseHoodieWriteClient writeClient, String inst
// Trigger compaction with suffixes based on the same instant time. This ensures that any future
// delta commits synced over will not have an instant time lesser than the last completed instant on the
// metadata table.
- final String compactionInstantTime = latestDeltaCommitTimeInMetadataTable + METADATA_COMPACTION_TIME_SUFFIX;
+ if (latestDeltaCommitTimeInMetadataTable.length() > MILLIS_INSTANT_TIMESTAMP_FORMAT_LENGTH) {
+ checkState(latestDeltaCommitTimeInMetadataTable.endsWith(METADATA_TABLE_INIT_TIME_SUFFIX),
Review Comment:
make sense to skip compaction in this case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1401341291
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a0838c73c28f77406b2865da1c0bf37cdd133822 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7740: [HUDI-5464] Suffix instant time for metadata partition init deltacommit
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7740:
URL: https://github.com/apache/hudi/pull/7740#issuecomment-1402782086
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14577",
"triggerID" : "a0838c73c28f77406b2865da1c0bf37cdd133822",
"triggerType" : "PUSH"
}, {
"hash" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581",
"triggerID" : "aefedbf6024c8de89e82ca227fbb60ac6647b824",
"triggerType" : "PUSH"
}, {
"hash" : "405071a2cab10908738a7c54269aff189ac13f47",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610",
"triggerID" : "405071a2cab10908738a7c54269aff189ac13f47",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* aefedbf6024c8de89e82ca227fbb60ac6647b824 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14581)
* 405071a2cab10908738a7c54269aff189ac13f47 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14610)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org