You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "linliu-code (via GitHub)" <gi...@apache.org> on 2025/08/20 00:42:40 UTC
[PR] [HUDI-9728] Extract and append record properties for commit and event time mergers [hudi]
linliu-code opened a new pull request, #13738:
URL: https://github.com/apache/hudi/pull/13738
### Change Logs
Previously for AwsDmsPayload, the custom delete key and marker are not set properly.
This change parses and append these properties during the initialization of record buffer.
### Impact
Make sure AwsDmsPayload can be deprecated properly.
### Risk level (write none, low medium or high below)
Low.
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for MOR [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206526377
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979)
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for MOR [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206604734
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979)
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213408523
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213851935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 0271547b357b6d5de0d347b535aebad29dabfe90 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046)
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292047911
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
I added one more delete batch that may trigger some different code path, which exposes the corner case for this payload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212684720
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 07cbf52a5f4558d9f579ca698f671e53b549d051 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021)
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213237516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
* f5eeb15d9989046dfc313b4680db5221b986c696 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040)
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3214013230
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties for commit and event time mergers [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203635611
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962)
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213141896
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
* f5eeb15d9989046dfc313b4680db5221b986c696 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040)
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2294490190
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +284,17 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
- tableConfig.getOrderingFieldsStr().orElse(null), tableVersion);
- recordMergeMode = triple.getLeft();
- mergeStrategyId = triple.getRight();
+ if (tableVersion.greaterThanOrEquals(HoodieTableVersion.NINE)) {
+ Map<String, String> mergeProperties = HoodieTableConfig.inferMergingConfigsForV9TableCreation(
+ null, writerPayloadClass.get(), null, tableConfig.getOrderingFieldsStr().orElse(null), tableVersion);
Review Comment:
Is this for expression payload which is not the same as table config on the writer side?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212864490
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213587198
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 0271547b357b6d5de0d347b535aebad29dabfe90 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211305093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 9b8ad369f57334002b6021a5d617fb4c071f344c Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011)
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3214657268
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
}, {
"hash" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8059",
"triggerID" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050)
* 95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8059)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292626432
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -162,6 +164,9 @@ protected HoodieTable(HoodieWriteConfig config, HoodieEngineContext context, Hoo
this.viewManager = getViewManager();
this.metaClient = metaClient;
this.taskContextSupplier = context.getTaskContextSupplier();
+
+ // Extract necessary properties from table config to write config.
+ reconcileWriteConfigs(metaClient, config);
Review Comment:
I checked the path for spark table, sounds like the write config within a table is the same one in the write handle. Maybe I missed something. I will do it in iohandle then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2294570341
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +284,17 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
- tableConfig.getOrderingFieldsStr().orElse(null), tableVersion);
- recordMergeMode = triple.getLeft();
- mergeStrategyId = triple.getRight();
+ if (tableVersion.greaterThanOrEquals(HoodieTableVersion.NINE)) {
+ Map<String, String> mergeProperties = HoodieTableConfig.inferMergingConfigsForV9TableCreation(
+ null, writerPayloadClass.get(), null, tableConfig.getOrderingFieldsStr().orElse(null), tableVersion);
Review Comment:
I think this is the scenarios that we discussion before: the payload class overwritten during write.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213770383
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 0271547b357b6d5de0d347b535aebad29dabfe90 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046)
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292743442
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -162,6 +164,9 @@ protected HoodieTable(HoodieWriteConfig config, HoodieEngineContext context, Hoo
this.viewManager = getViewManager();
this.metaClient = metaClient;
this.taskContextSupplier = context.getTaskContextSupplier();
+
+ // Extract necessary properties from table config to write config.
+ reconcileWriteConfigs(metaClient, config);
Review Comment:
I move it to HoodieBaseWriteClient.initTable(), which seems more inituitive. Meanwhile, we have to do it after the a potential upgrade/downgrade operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212622180
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 07cbf52a5f4558d9f579ca698f671e53b549d051 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021)
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212608393
<img width="1432" height="564" alt="image" src="https://github.com/user-attachments/assets/703fdfef-e065-49f7-b591-6102348afb59" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292669894
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -162,6 +164,9 @@ protected HoodieTable(HoodieWriteConfig config, HoodieEngineContext context, Hoo
this.viewManager = getViewManager();
this.metaClient = metaClient;
this.taskContextSupplier = context.getTaskContextSupplier();
+
+ // Extract necessary properties from table config to write config.
+ reconcileWriteConfigs(metaClient, config);
Review Comment:
Or the write config builder, because these options would be utilized in in-coming records merging, which is preceeding the write handles.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291963177
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +279,10 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
Review Comment:
Should the block below also update its usage of `inferMergingConfigsForPreV9Table`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213110398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
* f5eeb15d9989046dfc313b4680db5221b986c696 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203836475
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962)
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213441399
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 0271547b357b6d5de0d347b535aebad29dabfe90 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203853456
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962)
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292608062
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -275,11 +278,19 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
String mergeStrategyId = tableConfig.getRecordMergeStrategyId();
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
+ // TODO: simplify the following logic.
Review Comment:
Removed. Will do separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #13738:
URL: https://github.com/apache/hudi/pull/13738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292472542
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/TestFlinkWriteClients.java:
##########
@@ -107,7 +108,7 @@ void testRecordMergeConfigForEventTimeOrdering(boolean useLegacyConfig) throws E
assertThat(tableConfig.getRecordMergeMode(), is(RecordMergeMode.EVENT_TIME_ORDERING));
assertThat(tableConfig.getRecordMergeStrategyId(), is(HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID));
- assertThat(tableConfig.getPayloadClass(), is(EventTimeAvroPayload.class.getName()));
+ assertThat(tableConfig.getPayloadClass(), is(DefaultHoodieRecordPayload.class.getName()));
Review Comment:
This is not right.
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/TestFlinkWriteClients.java:
##########
@@ -152,7 +153,7 @@ void testRecordMergeConfigForPartialUpdate(int configOrdinal) throws Exception {
assertThat(tableConfig.getRecordMergeMode(), is(RecordMergeMode.EVENT_TIME_ORDERING));
assertThat(tableConfig.getRecordMergeStrategyId(), is(HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID));
- assertThat(tableConfig.getPayloadClass(), is(PartialUpdateAvroPayload.class.getName()));
+ assertThat(tableConfig.getPayloadClass(), is(DefaultHoodieRecordPayload.class.getName()));
Review Comment:
this is not right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211500477
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 9b8ad369f57334002b6021a5d617fb4c071f344c Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011)
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3214998804
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
}, {
"hash" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8059",
"triggerID" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8059)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211238436
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292048699
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -304,34 +464,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"MERGE_ON_READ",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
I need to add this back since this is for MOR, which is good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212067689
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016)
* 07cbf52a5f4558d9f579ca698f671e53b549d051 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212149521
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016)
* 07cbf52a5f4558d9f579ca698f671e53b549d051 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292626825
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -1883,7 +1883,7 @@ public void testPartialPayloadClass() throws Exception {
//now assert that hoodie.properties file now has updated payload class name
HoodieTableMetaClient metaClient = UtilHelpers.createMetaClient(jsc, dataSetBasePath, false);
- assertEquals(metaClient.getTableConfig().getPayloadClass(), PartialUpdateAvroPayload.class.getName());
+ assertEquals(metaClient.getTableConfig().getPayloadClass(), DefaultHoodieRecordPayload.class.getName());
Review Comment:
The write path is not completely moved away from payload yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206247521
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979)
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for MOR [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206875177
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3214469855
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
}, {
"hash" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8050)
* 95aa63c27c5d59e34e8e06da7ed6bc5224dbd9f1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292780491
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
Discussed with Tim offline, and Tim has a fix separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211282936
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980)
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 9b8ad369f57334002b6021a5d617fb4c071f344c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292067970
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -168,6 +184,162 @@ class TestPayloadDeprecationFlow extends SparkClientFunctionalTestHarness {
&& timeTravelDf.except(expectedTimeTravelDf).isEmpty)
}
+ @ParameterizedTest
+ @MethodSource(Array("providePayloadClassTestCases"))
+ def testMergerBuiltinPayloadFromTableCreationPath(tableType: String,
+ payloadClazz: String,
+ expectedConfigs: Map[String, String]): Unit = {
+ val opts: Map[String, String] = Map(
+ HoodieWriteConfig.WRITE_PAYLOAD_CLASS_NAME.key() -> payloadClazz,
+ HoodieMetadataConfig.ENABLE.key() -> "false")
+ val columns = Seq("ts", "_event_lsn", "rider", "driver", "fare", "Op", "_event_seq",
+ DebeziumConstants.FLATTENED_FILE_COL_NAME, DebeziumConstants.FLATTENED_POS_COL_NAME)
+ // 1. Add an insert.
+ val data = Seq(
+ (10, 1L, "rider-A", "driver-A", 19.10, "i", "10.1", 10, 1),
+ (10, 2L, "rider-B", "driver-B", 27.70, "i", "10.1", 10, 1),
+ (10, 3L, "rider-C", "driver-C", 33.90, "i", "10.1", 10, 1),
+ (10, 4L, "rider-D", "driver-D", 34.15, "i", "10.1", 10, 1),
+ (10, 5L, "rider-E", "driver-E", 17.85, "i", "10.1", 10, 1))
+ val inserts = spark.createDataFrame(data).toDF(columns: _*)
+ val orderingFields = if (payloadClazz.equals(classOf[MySqlDebeziumAvroPayload].getName)) {
+ "_event_bin_file,_event_pos"
+ } else {
+ "ts"
+ }
+ inserts.write.format("hudi").
+ option(RECORDKEY_FIELD.key(), "_event_lsn").
+ option(ORDERING_FIELDS.key(), orderingFields).
+ option(TABLE_TYPE.key(), tableType).
+ option(DataSourceWriteOptions.TABLE_NAME.key(), "test_table").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ options(opts).
+ mode(SaveMode.Overwrite).
+ save(basePath)
+ // Verify table was created successfully
+ var metaClient = HoodieTableMetaClient.builder()
+ .setBasePath(basePath)
+ .setConf(storageConf())
+ .build()
+ var tableConfig = metaClient.getTableConfig
+ // Verify table version is 9
+ assertEquals(9, tableConfig.getTableVersion.versionCode())
+ assertTrue(metaClient.getActiveTimeline.firstInstant().isPresent)
+ // Verify table properties
+ expectedConfigs.foreach { case (key, expectedValue) =>
+ if (expectedValue != null) {
+ assertEquals(expectedValue, tableConfig.getString(key), s"Config $key should be $expectedValue")
+ } else {
+ assertFalse(tableConfig.contains(key), s"Config $key should not be present")
+ }
+ }
+
+ // 2. Add an update.
+ val firstUpdateData = Seq(
+ (11, 1L, "rider-X", "driver-X", 19.10, "i", "11.1", 11, 1),
+ (12, 1L, "rider-X", "driver-X", 20.10, "D", "12.1", 12, 1),
+ (11, 2L, "rider-Y", "driver-Y", 27.70, "u", "11.1", 11, 1))
+ val firstUpdate = spark.createDataFrame(firstUpdateData).toDF(columns: _*)
+ firstUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ mode(SaveMode.Append).
+ save(basePath)
+ // Validate table version.
+ metaClient = HoodieTableMetaClient.reload(metaClient)
+ assertEquals(9, metaClient.getTableConfig.getTableVersion.versionCode())
+ val firstUpdateInstantTime = metaClient.getActiveTimeline.getInstants.get(1).requestedTime()
+
+
+ // 3. Add an update. This is expected to trigger the upgrade
+ val compactionEnabled = if (tableType.equals(HoodieTableType.MERGE_ON_READ.name())) {
+ "true"
+ } else {
+ "false"
+ }
+ val secondUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1),
+ (9, 4L, "rider-DD", "driver-DD", 34.15, "i", "9.1", 9, 1),
+ (12, 5L, "rider-EE", "driver-EE", 17.85, "i", "12.1", 12, 1))
+ val secondUpdate = spark.createDataFrame(secondUpdateData).toDF(columns: _*)
+ secondUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), compactionEnabled).
+ option(HoodieCompactionConfig.INLINE_COMPACT_NUM_DELTA_COMMITS.key(), "1").
+ mode(SaveMode.Append).
+ save(basePath)
+ // Validate table version as 9.
+ metaClient = HoodieTableMetaClient.reload(metaClient)
+ assertEquals(9, metaClient.getTableConfig.getTableVersion.versionCode())
+ assertEquals(payloadClazz, metaClient.getTableConfig.getLegacyPayloadClass)
+ val compactionInstants = metaClient.getActiveTimeline.getCommitsAndCompactionTimeline.getInstants
+ val foundCompaction = compactionInstants.stream().anyMatch(i => i.getAction.equals("commit"))
+ assertTrue(foundCompaction)
+
+ // 4. Add a trivial update to trigger payload class mismatch.
+ val thirdUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1))
+ val thirdUpdate = spark.createDataFrame(thirdUpdateData).toDF(columns: _*)
+ if (!payloadClazz.equals(classOf[MySqlDebeziumAvroPayload].getName)) {
+ assertThrows[HoodieException] {
+ thirdUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ option(HoodieCompactionConfig.INLINE_COMPACT_NUM_DELTA_COMMITS.key(), "1").
+ option(HoodieTableConfig.PAYLOAD_CLASS_NAME.key(),
+ classOf[MySqlDebeziumAvroPayload].getName).
+ mode(SaveMode.Append).
+ save(basePath)
+ }
+ }
+
+ // 5. Add a delete.
+ val fourthUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1),
Review Comment:
We should have some test that validates that the delete by the op field is working as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292116354
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -168,6 +184,162 @@ class TestPayloadDeprecationFlow extends SparkClientFunctionalTestHarness {
&& timeTravelDf.except(expectedTimeTravelDf).isEmpty)
}
+ @ParameterizedTest
+ @MethodSource(Array("providePayloadClassTestCases"))
+ def testMergerBuiltinPayloadFromTableCreationPath(tableType: String,
+ payloadClazz: String,
+ expectedConfigs: Map[String, String]): Unit = {
+ val opts: Map[String, String] = Map(
+ HoodieWriteConfig.WRITE_PAYLOAD_CLASS_NAME.key() -> payloadClazz,
+ HoodieMetadataConfig.ENABLE.key() -> "false")
+ val columns = Seq("ts", "_event_lsn", "rider", "driver", "fare", "Op", "_event_seq",
+ DebeziumConstants.FLATTENED_FILE_COL_NAME, DebeziumConstants.FLATTENED_POS_COL_NAME)
+ // 1. Add an insert.
+ val data = Seq(
+ (10, 1L, "rider-A", "driver-A", 19.10, "i", "10.1", 10, 1),
+ (10, 2L, "rider-B", "driver-B", 27.70, "i", "10.1", 10, 1),
+ (10, 3L, "rider-C", "driver-C", 33.90, "i", "10.1", 10, 1),
+ (10, 4L, "rider-D", "driver-D", 34.15, "i", "10.1", 10, 1),
+ (10, 5L, "rider-E", "driver-E", 17.85, "i", "10.1", 10, 1))
+ val inserts = spark.createDataFrame(data).toDF(columns: _*)
+ val orderingFields = if (payloadClazz.equals(classOf[MySqlDebeziumAvroPayload].getName)) {
+ "_event_bin_file,_event_pos"
+ } else {
+ "ts"
+ }
+ inserts.write.format("hudi").
+ option(RECORDKEY_FIELD.key(), "_event_lsn").
+ option(ORDERING_FIELDS.key(), orderingFields).
+ option(TABLE_TYPE.key(), tableType).
+ option(DataSourceWriteOptions.TABLE_NAME.key(), "test_table").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ options(opts).
+ mode(SaveMode.Overwrite).
+ save(basePath)
+ // Verify table was created successfully
+ var metaClient = HoodieTableMetaClient.builder()
+ .setBasePath(basePath)
+ .setConf(storageConf())
+ .build()
+ var tableConfig = metaClient.getTableConfig
+ // Verify table version is 9
+ assertEquals(9, tableConfig.getTableVersion.versionCode())
+ assertTrue(metaClient.getActiveTimeline.firstInstant().isPresent)
+ // Verify table properties
+ expectedConfigs.foreach { case (key, expectedValue) =>
+ if (expectedValue != null) {
+ assertEquals(expectedValue, tableConfig.getString(key), s"Config $key should be $expectedValue")
+ } else {
+ assertFalse(tableConfig.contains(key), s"Config $key should not be present")
+ }
+ }
+
+ // 2. Add an update.
+ val firstUpdateData = Seq(
+ (11, 1L, "rider-X", "driver-X", 19.10, "i", "11.1", 11, 1),
+ (12, 1L, "rider-X", "driver-X", 20.10, "D", "12.1", 12, 1),
+ (11, 2L, "rider-Y", "driver-Y", 27.70, "u", "11.1", 11, 1))
+ val firstUpdate = spark.createDataFrame(firstUpdateData).toDF(columns: _*)
+ firstUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ mode(SaveMode.Append).
+ save(basePath)
+ // Validate table version.
+ metaClient = HoodieTableMetaClient.reload(metaClient)
+ assertEquals(9, metaClient.getTableConfig.getTableVersion.versionCode())
+ val firstUpdateInstantTime = metaClient.getActiveTimeline.getInstants.get(1).requestedTime()
+
+
+ // 3. Add an update. This is expected to trigger the upgrade
+ val compactionEnabled = if (tableType.equals(HoodieTableType.MERGE_ON_READ.name())) {
+ "true"
+ } else {
+ "false"
+ }
+ val secondUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1),
+ (9, 4L, "rider-DD", "driver-DD", 34.15, "i", "9.1", 9, 1),
+ (12, 5L, "rider-EE", "driver-EE", 17.85, "i", "12.1", 12, 1))
+ val secondUpdate = spark.createDataFrame(secondUpdateData).toDF(columns: _*)
+ secondUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), compactionEnabled).
+ option(HoodieCompactionConfig.INLINE_COMPACT_NUM_DELTA_COMMITS.key(), "1").
+ mode(SaveMode.Append).
+ save(basePath)
+ // Validate table version as 9.
+ metaClient = HoodieTableMetaClient.reload(metaClient)
+ assertEquals(9, metaClient.getTableConfig.getTableVersion.versionCode())
+ assertEquals(payloadClazz, metaClient.getTableConfig.getLegacyPayloadClass)
+ val compactionInstants = metaClient.getActiveTimeline.getCommitsAndCompactionTimeline.getInstants
+ val foundCompaction = compactionInstants.stream().anyMatch(i => i.getAction.equals("commit"))
+ assertTrue(foundCompaction)
+
+ // 4. Add a trivial update to trigger payload class mismatch.
+ val thirdUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1))
+ val thirdUpdate = spark.createDataFrame(thirdUpdateData).toDF(columns: _*)
+ if (!payloadClazz.equals(classOf[MySqlDebeziumAvroPayload].getName)) {
+ assertThrows[HoodieException] {
+ thirdUpdate.write.format("hudi").
+ option(OPERATION.key(), "upsert").
+ option(HoodieCompactionConfig.INLINE_COMPACT.key(), "false").
+ option(HoodieCompactionConfig.INLINE_COMPACT_NUM_DELTA_COMMITS.key(), "1").
+ option(HoodieTableConfig.PAYLOAD_CLASS_NAME.key(),
+ classOf[MySqlDebeziumAvroPayload].getName).
+ mode(SaveMode.Append).
+ save(basePath)
+ }
+ }
+
+ // 5. Add a delete.
+ val fourthUpdateData = Seq(
+ (12, 3L, "rider-CC", "driver-CC", 33.90, "i", "12.1", 12, 1),
Review Comment:
We do. AwsDmsAvroPayload's result is different from other payloads since it uses the OP delete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292497399
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java:
##########
@@ -162,6 +164,9 @@ protected HoodieTable(HoodieWriteConfig config, HoodieEngineContext context, Hoo
this.viewManager = getViewManager();
this.metaClient = metaClient;
this.taskContextSupplier = context.getTaskContextSupplier();
+
+ // Extract necessary properties from table config to write config.
+ reconcileWriteConfigs(metaClient, config);
Review Comment:
should we fix the options in `HoodieIOHandle` instead of there, not sure whether the write config in hoodie table is actually the write config we use for write handles, we pass the write config separately for write handles actually.
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -1883,7 +1883,7 @@ public void testPartialPayloadClass() throws Exception {
//now assert that hoodie.properties file now has updated payload class name
HoodieTableMetaClient metaClient = UtilHelpers.createMetaClient(jsc, dataSetBasePath, false);
- assertEquals(metaClient.getTableConfig().getPayloadClass(), PartialUpdateAvroPayload.class.getName());
+ assertEquals(metaClient.getTableConfig().getPayloadClass(), DefaultHoodieRecordPayload.class.getName());
Review Comment:
not sure this is right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292609357
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/TestFlinkWriteClients.java:
##########
@@ -107,7 +108,7 @@ void testRecordMergeConfigForEventTimeOrdering(boolean useLegacyConfig) throws E
assertThat(tableConfig.getRecordMergeMode(), is(RecordMergeMode.EVENT_TIME_ORDERING));
assertThat(tableConfig.getRecordMergeStrategyId(), is(HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID));
- assertThat(tableConfig.getPayloadClass(), is(EventTimeAvroPayload.class.getName()));
+ assertThat(tableConfig.getPayloadClass(), is(DefaultHoodieRecordPayload.class.getName()));
Review Comment:
After payload deprecation for now. I think Tim is working on to remove all payload from the write path.
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/utils/TestFlinkWriteClients.java:
##########
@@ -152,7 +153,7 @@ void testRecordMergeConfigForPartialUpdate(int configOrdinal) throws Exception {
assertThat(tableConfig.getRecordMergeMode(), is(RecordMergeMode.EVENT_TIME_ORDERING));
assertThat(tableConfig.getRecordMergeStrategyId(), is(HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID));
- assertThat(tableConfig.getPayloadClass(), is(PartialUpdateAvroPayload.class.getName()));
+ assertThat(tableConfig.getPayloadClass(), is(DefaultHoodieRecordPayload.class.getName()));
Review Comment:
The same as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213096752
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
* f5eeb15d9989046dfc313b4680db5221b986c696 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203706991
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962)
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for MOR [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206281327
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979)
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for MOR [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206332298
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979)
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 046c43e3f4ba383cbc7948e1e55f9703019b158f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3206240529
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
* b5185e48c23d2fba5882b196f0a15f6e3edbd6cf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291524235
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +279,10 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
Review Comment:
This was a bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291571322
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
> Caused by: java.lang.NullPointerException
at org.apache.hudi.common.util.FileFormatUtils$HoodieKeyIterator.lambda$new$3(FileFormatUtils.java:388)
at org.apache.hudi.common.util.FileFormatUtils$HoodieKeyIterator.next(FileFormatUtils.java:409)
at org.apache.hudi.common.util.FileFormatUtils$HoodieKeyIterator.next(FileFormatUtils.java:370)
at org.apache.hudi.common.util.collection.MappingIterator.next(MappingIterator.java:44)
at org.apache.hudi.common.util.collection.MappingIterator.next(MappingIterator.java:44)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291565443
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
There is an irrelevant bug. Disable it for now. Will fix in a separate minor PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291565443
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
There is an irrelevant bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292005810
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +279,10 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
Review Comment:
For read branch below, v9 tables are supposed to have all merge properties set properly at creation or during upgrade. So no inference should be attempted.
For this write branch, the merge properties can be overwritten by temporary payload classes, merge mode or strategy id; therefore, we need to run the inference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291524235
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -276,10 +279,10 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
if (isIngestion && writerPayloadClass.map(className -> !className.equals(tableConfig.getPayloadClass())).orElse(false)) {
- Triple<RecordMergeMode, String, String> triple = HoodieTableConfig.inferMergingConfigsForWrites(null, writerPayloadClass.get(), null,
Review Comment:
This was a bug, since the payload class can be the deprecated payload class, and the final record merge mode should be different from CUSTOM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212317357
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 07cbf52a5f4558d9f579ca698f671e53b549d051 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213374486
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* f5eeb15d9989046dfc313b4680db5221b986c696 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040)
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Extract and append record properties for commit and event time mergers [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203562711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Extract and append record properties for commit and event time mergers [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203578624
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e7f05bd181add5427e4fc6adc2bf44a0c8b4741d Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3212712587
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 07cbf52a5f4558d9f579ca698f671e53b549d051 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021)
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 55210a3da75e93c514bec5f1dd6fb4a961503f3b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291566089
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -304,34 +464,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"MERGE_ON_READ",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
There is an irrelevant bug. Disable it for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211436972
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 9b8ad369f57334002b6021a5d617fb4c071f344c Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011)
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2292444860
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java:
##########
@@ -275,11 +278,19 @@ private void initRecordMerger(TypedProperties properties, boolean isIngestion) {
String mergeStrategyId = tableConfig.getRecordMergeStrategyId();
HoodieTableVersion tableVersion = tableConfig.getTableVersion();
// If the provided payload class differs from the table's payload class, we need to infer the correct merging behavior.
+ // TODO: simplify the following logic.
Review Comment:
Can we do this as part of this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213435123
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3211681933
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* d7e8b7f98be77c53cacb53ceea1ce7af953ffe02 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13738:
URL: https://github.com/apache/hudi/pull/13738#discussion_r2291957232
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestPayloadDeprecationFlow.scala:
##########
@@ -234,34 +400,27 @@ object TestPayloadDeprecationFlow {
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "EVENT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[DefaultHoodieRecordPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)),
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.EVENT_TIME_BASED_MERGE_STRATEGY_UUID)
+ ),
Arguments.of(
"COPY_ON_WRITE",
classOf[OverwriteWithLatestAvroPayload].getName,
Map(
HoodieTableConfig.RECORD_MERGE_MODE.key() -> "COMMIT_TIME_ORDERING",
HoodieTableConfig.LEGACY_PAYLOAD_CLASS_NAME.key() -> classOf[OverwriteWithLatestAvroPayload].getName,
- HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID
- )
+ HoodieTableConfig.RECORD_MERGE_STRATEGY_ID.key() -> HoodieRecordMerger.COMMIT_TIME_BASED_MERGE_STRATEGY_UUID)
),
- Arguments.of(
Review Comment:
Why does this change trigger this issue though?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] set record merge properties properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3203988048
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 79fa302b06146a86a88f5febae2c636f7ef0c136 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9728] Set record merge properties properly for write and read paths [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13738:
URL: https://github.com/apache/hudi/pull/13738#issuecomment-3213708852
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7962",
"triggerID" : "e7f05bd181add5427e4fc6adc2bf44a0c8b4741d",
"triggerType" : "PUSH"
}, {
"hash" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e1545a3d2bbaeb89c65fa8605302db7a52528e35",
"triggerType" : "PUSH"
}, {
"hash" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7964",
"triggerID" : "79fa302b06146a86a88f5febae2c636f7ef0c136",
"triggerType" : "PUSH"
}, {
"hash" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7979",
"triggerID" : "b5185e48c23d2fba5882b196f0a15f6e3edbd6cf",
"triggerType" : "PUSH"
}, {
"hash" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "640228d134a7686ae943a1a2a342e999817c5cbf",
"triggerType" : "PUSH"
}, {
"hash" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1109ee0b540a281697ed6f7af40774e8f8e6a87",
"triggerType" : "PUSH"
}, {
"hash" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=7980",
"triggerID" : "046c43e3f4ba383cbc7948e1e55f9703019b158f",
"triggerType" : "PUSH"
}, {
"hash" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9651c2640c08f270ec336aaaf5a60cda8961bf2d",
"triggerType" : "PUSH"
}, {
"hash" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8011",
"triggerID" : "9b8ad369f57334002b6021a5d617fb4c071f344c",
"triggerType" : "PUSH"
}, {
"hash" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8016",
"triggerID" : "d7e8b7f98be77c53cacb53ceea1ce7af953ffe02",
"triggerType" : "PUSH"
}, {
"hash" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8021",
"triggerID" : "07cbf52a5f4558d9f579ca698f671e53b549d051",
"triggerType" : "PUSH"
}, {
"hash" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29e3d6f076ecdcb2b45c801958e901395f20feca",
"triggerType" : "PUSH"
}, {
"hash" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8029",
"triggerID" : "55210a3da75e93c514bec5f1dd6fb4a961503f3b",
"triggerType" : "PUSH"
}, {
"hash" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8040",
"triggerID" : "f5eeb15d9989046dfc313b4680db5221b986c696",
"triggerType" : "PUSH"
}, {
"hash" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043",
"triggerID" : "7ce0eb9087185006460906d23adf9bd94f4e6c80",
"triggerType" : "PUSH"
}, {
"hash" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3d88102695f91202a152ac6714d6aab3efd8b7bb",
"triggerType" : "PUSH"
}, {
"hash" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046",
"triggerID" : "0271547b357b6d5de0d347b535aebad29dabfe90",
"triggerType" : "PUSH"
}, {
"hash" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "14d9c3645c188aa2a2605e326ce1f90b11178e89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e1545a3d2bbaeb89c65fa8605302db7a52528e35 UNKNOWN
* 640228d134a7686ae943a1a2a342e999817c5cbf UNKNOWN
* d1109ee0b540a281697ed6f7af40774e8f8e6a87 UNKNOWN
* 9651c2640c08f270ec336aaaf5a60cda8961bf2d UNKNOWN
* 29e3d6f076ecdcb2b45c801958e901395f20feca UNKNOWN
* 7ce0eb9087185006460906d23adf9bd94f4e6c80 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8043)
* 3d88102695f91202a152ac6714d6aab3efd8b7bb UNKNOWN
* 0271547b357b6d5de0d347b535aebad29dabfe90 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=8046)
* 14d9c3645c188aa2a2605e326ce1f90b11178e89 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org