You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/02/23 18:21:45 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
alexeykudinkin opened a new pull request #4888:
URL: https://github.com/apache/hudi/pull/4888
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
Refactoring `MergeOnReadRDD` to
- Avoid duplication
- Enable optimization to avoid reading base-file with full-schema, fetching only projected columns
## Brief change log
- Unified MOR record iteration logic w/in 3 iterators extending each other
- Added optimization to consider whether a) virtual keys are used and b) whether non-default RecordPayload class is used to determine whether we can do projected read from base-file instead of full-schema one
## Verify this pull request
This pull request is already covered by existing tests, such as *(please describe tests)*.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065975774
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069576250
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071666333
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068543109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075643139
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
* 366ec612b7a31fe375730161f470a9135a7d3ff9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075684390
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075690787
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1079158087
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 7bfcb40af344c2589311f0565384387a172f9304 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078679289
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
* 7bfcb40af344c2589311f0565384387a172f9304 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r834964133
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
Review comment:
Frankly, there's not a lot of added value in this Trait, given that its API will be practically identical to that one of `SparkAdapterSupport`:
```
def createAvroSerializer(st: StructType, schema: Schema)
```
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
Review comment:
Good call!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078695230
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
* 7bfcb40af344c2589311f0565384387a172f9304 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050322824
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074416349
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072901323
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074365979
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074835414
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072883576
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071514052
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071516654
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072003035
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067396258
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069518757
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049082728
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065838328
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
* b8272f7316953648a46fafe5065965a4552a7e0e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r835399807
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +76,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
+
case logFileOnlySplit if logFileOnlySplit.dataFile.isEmpty =>
- logFileIterator(logFileOnlySplit, getConfig)
- case skipMergeSplit if skipMergeSplit.mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
- skipMergeFileIterator(skipMergeSplit, requiredSchemaFileReader(skipMergeSplit.dataFile.get), getConfig)
- case payloadCombineSplit
- if payloadCombineSplit.mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
- payloadCombineFileIterator(payloadCombineSplit, fullSchemaFileReader(payloadCombineSplit.dataFile.get),
- getConfig)
+ new LogFileIterator(logFileOnlySplit, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
+ val baseFileIterator = requiredSchemaFileReader.apply(split.dataFile.get)
+ new SkipMergeIterator(split, baseFileIterator, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
+ val (baseFileIterator, schema) = readBaseFile(split)
+ new RecordMergingFileIterator(split, baseFileIterator, schema, getConfig)
+
case _ => throw new HoodieException(s"Unable to select an Iterator to read the Hoodie MOR File Split for " +
s"file path: ${mergeOnReadPartition.split.dataFile.get.filePath}" +
s"log paths: ${mergeOnReadPartition.split.logFiles.toString}" +
- s"hoodie table path: ${mergeOnReadPartition.split.tablePath}" +
+ s"hoodie table path: ${tableState.tablePath}" +
s"spark partition Index: ${mergeOnReadPartition.index}" +
- s"merge type: ${mergeOnReadPartition.split.mergeType}")
+ s"merge type: ${mergeType}")
}
+
if (iter.isInstanceOf[Closeable]) {
// register a callback to close logScanner which will be executed on task completion.
// when tasks finished, this method will be called, and release resources.
Option(TaskContext.get()).foreach(_.addTaskCompletionListener[Unit](_ => iter.asInstanceOf[Closeable].close()))
}
+
iter
}
+ private def readBaseFile(split: HoodieMergeOnReadFileSplit): (Iterator[InternalRow], HoodieTableSchema) = {
+ // NOTE: This is an optimization making sure that even for MOR tables we fetch absolute minimum
+ // of the stored data possible, while still properly executing corresponding relation's semantic
+ // and meet the query's requirements.
+ //
+ // Here we assume that iff queried table
+ // a) It does use one of the standard (and whitelisted) Record Payload classes
+ // then we can avoid reading and parsing the records w/ _full_ schema, and instead only
+ // rely on projected one, nevertheless being able to perform merging correctly
+ if (!whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
Review comment:
Not sure i follow how you propose to change this expr?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
Review comment:
I inlined `maxCompactionMemoryInBytes` into `HoodieMergeOnReadRDD` (previously was passed in as an arg), but i don't think it makes sense to eliminate it from this arg line here -- it's not a simple field access but requires quite some computation.
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +139,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
+ val tablePath = tableState.tablePath
+ val fs = FSUtils.getFs(tablePath, hadoopConf)
- if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+ if (HoodieTableMetadata.isMetadataTable(tablePath)) {
val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
- val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+ val dataTableBasePath = getDataTableBasePathFromMetadataTable(tablePath)
val metadataTable = new HoodieBackedTableMetadata(
- new HoodieLocalEngineContext(config), metadataConfig,
+ new HoodieLocalEngineContext(hadoopConf), metadataConfig,
dataTableBasePath,
- config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+ hadoopConf.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
// NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
// of indirection among MT partitions)
- val relativePartitionPath = getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+ val relativePartitionPath = getRelativePartitionPath(new Path(tablePath), partitionPath)
metadataTable.getLogRecordScanner(logFiles.asJava, relativePartitionPath).getLeft
} else {
val logRecordScannerBuilder = HoodieMergedLogRecordScanner.newBuilder()
.withFileSystem(fs)
- .withBasePath(split.tablePath)
- .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)
+ .withBasePath(tablePath)
+ .withLogFilePaths(logFiles.map(logFile => getFilePath(logFile.getPath)).asJava)
.withReaderSchema(logSchema)
- .withLatestInstantTime(split.latestCommit)
+ .withLatestInstantTime(tableState.latestCommitTimestamp)
.withReadBlocksLazily(
- Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
+ Try(hadoopConf.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
.getOrElse(false))
.withReverseReader(false)
.withBufferSize(
- config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
+ hadoopConf.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
- .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
+ .withMaxMemorySizeInBytes(maxCompactionMemoryInBytes)
.withSpillableMapBasePath(
- config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
+ hadoopConf.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
if (logFiles.nonEmpty) {
- logRecordScannerBuilder.withPartition(getRelativePartitionPath(new Path(split.tablePath), logFiles.head.getPath.getParent))
+ logRecordScannerBuilder.withPartition(
+ getRelativePartitionPath(new Path(tableState.tablePath), logFiles.head.getPath.getParent))
}
logRecordScannerBuilder.build()
}
}
+ /**
+ * Projects provided instance of [[InternalRow]] into provided schema, assuming that the
+ * the schema of the original row is strictly a superset of the given one
+ */
+ private def projectRowUnsafe(row: InternalRow,
+ projectedSchema: StructType,
+ ordinals: Seq[Int]): InternalRow = {
+ val projectedRow = new SpecificInternalRow(projectedSchema)
+ var curIndex = 0
+ projectedSchema.zip(ordinals).foreach { case (field, pos) =>
+ val curField = if (row.isNullAt(pos)) {
+ null
+ } else {
+ row.get(pos, field.dataType)
+ }
+ projectedRow.update(curIndex, curField)
+ curIndex += 1
+ }
+ projectedRow
+ }
+
+ /**
+ * Projects provided instance of [[IndexedRecord]] into provided schema, assuming that the
+ * the schema of the original row is strictly a superset of the given one
+ */
+ def projectAvroUnsafe(record: IndexedRecord,
+ projectedSchema: Schema,
+ ordinals: List[Int],
+ recordBuilder: GenericRecordBuilder): GenericRecord = {
+ val fields = projectedSchema.getFields.asScala
+ assert(fields.length == ordinals.length)
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068576826
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065975774
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069518757
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832627036
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileScanRDD.scala
##########
@@ -20,64 +20,15 @@ package org.apache.hudi
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.catalyst.InternalRow
-import org.apache.spark.sql.execution.QueryExecutionException
-import org.apache.spark.sql.execution.datasources.{FilePartition, PartitionedFile, SchemaColumnConvertNotSupportedException}
-import org.apache.spark.{Partition, TaskContext}
+import org.apache.spark.sql.execution.datasources.{FilePartition, FileScanRDD, PartitionedFile}
case class HoodieBaseFileSplit(filePartition: FilePartition) extends HoodieFileSplit
-/**
- * TODO eval if we actually need it
- */
class HoodieFileScanRDD(@transient private val sparkSession: SparkSession,
readFunction: PartitionedFile => Iterator[InternalRow],
@transient fileSplits: Seq[HoodieBaseFileSplit])
- extends HoodieUnsafeRDD(sparkSession.sparkContext) {
-
- override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
Review comment:
Pretty much
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075684390
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r831638796
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BaseFileOnlyRelation.scala
##########
@@ -52,6 +53,14 @@ class BaseFileOnlyRelation(sqlContext: SQLContext,
override type FileSplit = HoodieBaseFileSplit
+ override lazy val mandatoryColumns: Seq[String] = {
+ if (isMetadataTable(metaClient)) {
Review comment:
Good call, this actually not required here
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -79,13 +84,30 @@ abstract class HoodieBaseRelation(val sqlContext: SQLContext,
protected lazy val basePath: String = metaClient.getBasePath
- // If meta fields are enabled, always prefer key from the meta field as opposed to user-specified one
- // NOTE: This is historical behavior which is preserved as is
+ // NOTE: Record key-field is assumed singular here due to the either of
Review comment:
This actually refers to metadata fields, not MT. Will amend
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074549109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074547340
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074659089
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049089113
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050359771
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050324902
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074444305
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078558718
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065839129
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064721168
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069513264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
* c7321c52d6f6d883e40d7331c782d0ea265172b6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078676722
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832632309
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
Review comment:
The problem with `DefaultHoodieRecordPayload` that it's also requiring `PAYLOAD_EVENT_TIME_FIELD_PROP_KEY` field. In the future when we migrate to unified Merge API we would allow particular configured Merge Engine to expose which fields are required for it and then make Relation load those. For now, i opted in to keep things simple here until we reach that point. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1079158087
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 7bfcb40af344c2589311f0565384387a172f9304 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050320927
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050387755
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068543109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074613546
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074726932
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074371687
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074363130
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074726932
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071518711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
* 59ff930ff76d233229c9384215ff4158a81eecd5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072882599
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072883576
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1079198498
@alexeykudinkin landing this as the CI passed before "Tidying up" commit. Also basic multi spark version tests passed in GA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r834964979
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -380,4 +422,17 @@ private object HoodieMergeOnReadRDD {
case (nullable, _) => nullable
}
}
+
+ trait AvroDeserializerSupport extends SparkAdapterSupport {
Review comment:
Nah, not really -- it requires you to provide required schema, so pretty much falls back into the same bucket as `createAvroSerializer` (in my comment above).
The only reason i've extracted it to a separate Trait is to trim down Log Iterator impl, moving out all the stuff which isn't really tied to iteration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049089113
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064723457
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065838328
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
* b8272f7316953648a46fafe5065965a4552a7e0e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065988355
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064722341
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065838782
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067335814
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049085839
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049085839
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072004489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072054380
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065988355
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069576250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071518711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
* 59ff930ff76d233229c9384215ff4158a81eecd5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072003035
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071666333
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069576250
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068576826
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074613546
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074659089
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078676722
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078679289
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
* 7bfcb40af344c2589311f0565384387a172f9304 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1079169032
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
}, {
"hash" : "54be92b4da1fe0e656dfb7a103287b5a83549134",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "54be92b4da1fe0e656dfb7a103287b5a83549134",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 7bfcb40af344c2589311f0565384387a172f9304 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354)
* 54be92b4da1fe0e656dfb7a103287b5a83549134 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064722341
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064721168
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065975375
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
* 599584e29b5fbafa702815b8a441ecaef2ed6009 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071514052
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064831876
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065838782
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065975375
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
* 599584e29b5fbafa702815b8a441ecaef2ed6009 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832116327
##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -206,17 +206,6 @@ object AvroConversionUtils {
SchemaConverters.toSqlType(avroSchema).dataType.asInstanceOf[StructType]
}
- def buildAvroRecordBySchema(record: IndexedRecord,
Review comment:
this method not used? would be helpful to have a comment here to clarify
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
+
case logFileOnlySplit if logFileOnlySplit.dataFile.isEmpty =>
- logFileIterator(logFileOnlySplit, getConfig)
- case skipMergeSplit if skipMergeSplit.mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
- skipMergeFileIterator(skipMergeSplit, requiredSchemaFileReader(skipMergeSplit.dataFile.get), getConfig)
- case payloadCombineSplit
- if payloadCombineSplit.mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
- payloadCombineFileIterator(payloadCombineSplit, fullSchemaFileReader(payloadCombineSplit.dataFile.get),
- getConfig)
+ new LogFileIterator(logFileOnlySplit, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
+ val baseFileIterator = requiredSchemaFileReader.apply(split.dataFile.get)
+ new SkipMergeIterator(split, baseFileIterator, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
+ val (baseFileIterator, schema) = readBaseFile(split)
+ new RecordMergingFileIterator(split, baseFileIterator, schema, getConfig)
+
case _ => throw new HoodieException(s"Unable to select an Iterator to read the Hoodie MOR File Split for " +
s"file path: ${mergeOnReadPartition.split.dataFile.get.filePath}" +
s"log paths: ${mergeOnReadPartition.split.logFiles.toString}" +
- s"hoodie table path: ${mergeOnReadPartition.split.tablePath}" +
+ s"hoodie table path: ${tableState.tablePath}" +
s"spark partition Index: ${mergeOnReadPartition.index}" +
- s"merge type: ${mergeOnReadPartition.split.mergeType}")
+ s"merge type: ${mergeType}")
}
+
if (iter.isInstanceOf[Closeable]) {
// register a callback to close logScanner which will be executed on task completion.
// when tasks finished, this method will be called, and release resources.
Option(TaskContext.get()).foreach(_.addTaskCompletionListener[Unit](_ => iter.asInstanceOf[Closeable].close()))
}
+
iter
}
+ private def readBaseFile(split: HoodieMergeOnReadFileSplit): (Iterator[InternalRow], HoodieTableSchema) = {
+ // NOTE: This is an optimization making sure that even for MOR tables we fetch absolute minimum
+ // of the stored data possible, while still properly executing corresponding relation's semantic
+ // and meet the query's requirements.
+ //
+ // Here we assume that iff queried table
+ // a) It does use one of the standard (and whitelisted) Record Payload classes
+ // then we can avoid reading and parsing the records w/ _full_ schema, and instead only
+ // rely on projected one, nevertheless being able to perform merging correctly
+ if (!whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
+ (fullSchemaFileReader(split.dataFile.get), tableSchema)
Review comment:
looks better with `fullSchema` to distinguish with `requiredSchema`
```suggestion
(fullSchemaFileReader(split.dataFile.get), fullSchema)
```
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
Review comment:
maybe make `AvroDeserializerSupport` provide a serializer and call `AvroSerDeSupport` ?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -380,4 +422,17 @@ private object HoodieMergeOnReadRDD {
case (nullable, _) => nullable
}
}
+
+ trait AvroDeserializerSupport extends SparkAdapterSupport {
Review comment:
this looks like a generic trait.. maybe place in a more common class?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -79,13 +83,30 @@ abstract class HoodieBaseRelation(val sqlContext: SQLContext,
protected lazy val basePath: String = metaClient.getBasePath
- // If meta fields are enabled, always prefer key from the meta field as opposed to user-specified one
- // NOTE: This is historical behavior which is preserved as is
+ // NOTE: Record key-field is assumed singular here due to the either of
+ // - In case Hudi's meta fields are enabled: record key will be pre-materialized (stored) as part
+ // of the record's payload (as part of the Hudi's metadata)
+ // - In case Hudi's meta fields are disabled (virtual keys): in that case record has to bear _single field_
+ // identified as its (unique) primary key w/in its payload (this is a limitation of [[SimpleKeyGenerator]],
+ // which is the only [[KeyGenerator]] permitted for virtual-keys payloads)
protected lazy val recordKeyField: String =
- if (tableConfig.populateMetaFields()) HoodieRecord.RECORD_KEY_METADATA_FIELD
- else tableConfig.getRecordKeyFieldProp
+ if (tableConfig.populateMetaFields()) {
+ HoodieRecord.RECORD_KEY_METADATA_FIELD
+ } else {
+ val keyFields = tableConfig.getRecordKeyFields.get()
+ checkState(keyFields.length == 1)
+ keyFields.head
+ }
- protected lazy val preCombineFieldOpt: Option[String] = getPrecombineFieldProperty
+ protected lazy val preCombineFieldOpt: Option[String] =
+ Option(tableConfig.getPreCombineField)
+ .orElse(optParams.get(DataSourceWriteOptions.PRECOMBINE_FIELD.key)) match {
+ // NOTE: This is required to compensate for cases when empty string is used to stub
+ // property value to avoid it being set with the default value
+ // TODO(HUDI-3456) cleanup
+ case Some(f) if !StringUtils.isNullOrEmpty(f) => Some(f)
Review comment:
have seen this negation many other places. It would be more readable to add the other util .
```suggestion
case Some(f) if StringUtils.nonEmpty(f) => Some(f)
```
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
Review comment:
should this include `DefaultHoodieRecordPayload` ?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
Review comment:
let's always call baseFile instead of dataFile, while you're at it
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
Review comment:
why still have `maxCompactionMemoryInBytes` as an arg, which can be retrieved from `hadoopConf` ? having >5 args makes the API harder to use
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieFileScanRDD.scala
##########
@@ -20,64 +20,15 @@ package org.apache.hudi
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.catalyst.InternalRow
-import org.apache.spark.sql.execution.QueryExecutionException
-import org.apache.spark.sql.execution.datasources.{FilePartition, PartitionedFile, SchemaColumnConvertNotSupportedException}
-import org.apache.spark.{Partition, TaskContext}
+import org.apache.spark.sql.execution.datasources.{FilePartition, FileScanRDD, PartitionedFile}
case class HoodieBaseFileSplit(filePartition: FilePartition) extends HoodieFileSplit
-/**
- * TODO eval if we actually need it
- */
class HoodieFileScanRDD(@transient private val sparkSession: SparkSession,
readFunction: PartitionedFile => Iterator[InternalRow],
@transient fileSplits: Seq[HoodieBaseFileSplit])
- extends HoodieUnsafeRDD(sparkSession.sparkContext) {
-
- override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
Review comment:
this was spark code copied over?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832942841
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -104,7 +104,7 @@ abstract class HoodieBaseRelation(val sqlContext: SQLContext,
// NOTE: This is required to compensate for cases when empty string is used to stub
// property value to avoid it being set with the default value
// TODO(HUDI-3456) cleanup
- case Some(f) if !StringUtils.isNullOrEmpty(f) => Some(f)
+ case Some(f) if StringUtils.nonEmpty(f) => Some(f)
Review comment:
this method `nonEmpty()` does not exist yet. i should have clarified it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832626454
##########
File path: hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/AvroConversionUtils.scala
##########
@@ -206,17 +206,6 @@ object AvroConversionUtils {
SchemaConverters.toSqlType(avroSchema).dataType.asInstanceOf[StructType]
}
- def buildAvroRecordBySchema(record: IndexedRecord,
Review comment:
Correct. I'm only removing methods that aren't used anymore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832926186
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
Review comment:
sounds ok to make it work with OverwriteWithLatestAvroPayload which is the current default, covering most cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074550711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074363130
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074550711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074549109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074743538
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078544616
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
* c68818933f0a440f928feeccf33e5704fe0a165c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078544616
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
* c68818933f0a440f928feeccf33e5704fe0a165c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r835312918
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +76,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
+
case logFileOnlySplit if logFileOnlySplit.dataFile.isEmpty =>
- logFileIterator(logFileOnlySplit, getConfig)
- case skipMergeSplit if skipMergeSplit.mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
- skipMergeFileIterator(skipMergeSplit, requiredSchemaFileReader(skipMergeSplit.dataFile.get), getConfig)
- case payloadCombineSplit
- if payloadCombineSplit.mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
- payloadCombineFileIterator(payloadCombineSplit, fullSchemaFileReader(payloadCombineSplit.dataFile.get),
- getConfig)
+ new LogFileIterator(logFileOnlySplit, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
+ val baseFileIterator = requiredSchemaFileReader.apply(split.dataFile.get)
+ new SkipMergeIterator(split, baseFileIterator, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
+ val (baseFileIterator, schema) = readBaseFile(split)
+ new RecordMergingFileIterator(split, baseFileIterator, schema, getConfig)
+
case _ => throw new HoodieException(s"Unable to select an Iterator to read the Hoodie MOR File Split for " +
s"file path: ${mergeOnReadPartition.split.dataFile.get.filePath}" +
s"log paths: ${mergeOnReadPartition.split.logFiles.toString}" +
- s"hoodie table path: ${mergeOnReadPartition.split.tablePath}" +
+ s"hoodie table path: ${tableState.tablePath}" +
s"spark partition Index: ${mergeOnReadPartition.index}" +
- s"merge type: ${mergeOnReadPartition.split.mergeType}")
+ s"merge type: ${mergeType}")
}
+
if (iter.isInstanceOf[Closeable]) {
// register a callback to close logScanner which will be executed on task completion.
// when tasks finished, this method will be called, and release resources.
Option(TaskContext.get()).foreach(_.addTaskCompletionListener[Unit](_ => iter.asInstanceOf[Closeable].close()))
}
+
iter
}
+ private def readBaseFile(split: HoodieMergeOnReadFileSplit): (Iterator[InternalRow], HoodieTableSchema) = {
+ // NOTE: This is an optimization making sure that even for MOR tables we fetch absolute minimum
+ // of the stored data possible, while still properly executing corresponding relation's semantic
+ // and meet the query's requirements.
+ //
+ // Here we assume that iff queried table
+ // a) It does use one of the standard (and whitelisted) Record Payload classes
+ // then we can avoid reading and parsing the records w/ _full_ schema, and instead only
+ // rely on projected one, nevertheless being able to perform merging correctly
+ if (!whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
Review comment:
/nit I'd prefer flip the condition to avoid logical negation; less mind processing
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -380,4 +425,17 @@ private object HoodieMergeOnReadRDD {
case (nullable, _) => nullable
}
}
+
+ trait AvroDeserializerSupport extends SparkAdapterSupport {
+ protected val requiredAvroSchema: Schema
+ protected val requiredStructTypeSchema: StructType
+
+ private lazy val deserializer: HoodieAvroDeserializer =
+ sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredStructTypeSchema)
+
+ protected def deserialize(avroRecord: GenericRecord): InternalRow = {
+ assert(avroRecord.getSchema.getFields.size() == requiredStructTypeSchema.fields.length)
Review comment:
use `checkstate` with a message ?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +139,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
+ val tablePath = tableState.tablePath
+ val fs = FSUtils.getFs(tablePath, hadoopConf)
- if (HoodieTableMetadata.isMetadataTable(split.tablePath)) {
+ if (HoodieTableMetadata.isMetadataTable(tablePath)) {
val metadataConfig = HoodieMetadataConfig.newBuilder().enable(true).build()
- val dataTableBasePath = getDataTableBasePathFromMetadataTable(split.tablePath)
+ val dataTableBasePath = getDataTableBasePathFromMetadataTable(tablePath)
val metadataTable = new HoodieBackedTableMetadata(
- new HoodieLocalEngineContext(config), metadataConfig,
+ new HoodieLocalEngineContext(hadoopConf), metadataConfig,
dataTableBasePath,
- config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
+ hadoopConf.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP, HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
// NOTE: In case of Metadata Table partition path equates to partition name (since there's just one level
// of indirection among MT partitions)
- val relativePartitionPath = getRelativePartitionPath(new Path(split.tablePath), getPartitionPath(split))
+ val relativePartitionPath = getRelativePartitionPath(new Path(tablePath), partitionPath)
metadataTable.getLogRecordScanner(logFiles.asJava, relativePartitionPath).getLeft
} else {
val logRecordScannerBuilder = HoodieMergedLogRecordScanner.newBuilder()
.withFileSystem(fs)
- .withBasePath(split.tablePath)
- .withLogFilePaths(split.logFiles.get.map(logFile => getFilePath(logFile.getPath)).asJava)
+ .withBasePath(tablePath)
+ .withLogFilePaths(logFiles.map(logFile => getFilePath(logFile.getPath)).asJava)
.withReaderSchema(logSchema)
- .withLatestInstantTime(split.latestCommit)
+ .withLatestInstantTime(tableState.latestCommitTimestamp)
.withReadBlocksLazily(
- Try(config.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
+ Try(hadoopConf.get(HoodieRealtimeConfig.COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP,
HoodieRealtimeConfig.DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED).toBoolean)
.getOrElse(false))
.withReverseReader(false)
.withBufferSize(
- config.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
+ hadoopConf.getInt(HoodieRealtimeConfig.MAX_DFS_STREAM_BUFFER_SIZE_PROP,
HoodieRealtimeConfig.DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE))
- .withMaxMemorySizeInBytes(split.maxCompactionMemoryInBytes)
+ .withMaxMemorySizeInBytes(maxCompactionMemoryInBytes)
.withSpillableMapBasePath(
- config.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
+ hadoopConf.get(HoodieRealtimeConfig.SPILLABLE_MAP_BASE_PATH_PROP,
HoodieRealtimeConfig.DEFAULT_SPILLABLE_MAP_BASE_PATH))
if (logFiles.nonEmpty) {
- logRecordScannerBuilder.withPartition(getRelativePartitionPath(new Path(split.tablePath), logFiles.head.getPath.getParent))
+ logRecordScannerBuilder.withPartition(
+ getRelativePartitionPath(new Path(tableState.tablePath), logFiles.head.getPath.getParent))
}
logRecordScannerBuilder.build()
}
}
+ /**
+ * Projects provided instance of [[InternalRow]] into provided schema, assuming that the
+ * the schema of the original row is strictly a superset of the given one
+ */
+ private def projectRowUnsafe(row: InternalRow,
+ projectedSchema: StructType,
+ ordinals: Seq[Int]): InternalRow = {
+ val projectedRow = new SpecificInternalRow(projectedSchema)
+ var curIndex = 0
+ projectedSchema.zip(ordinals).foreach { case (field, pos) =>
+ val curField = if (row.isNullAt(pos)) {
+ null
+ } else {
+ row.get(pos, field.dataType)
+ }
+ projectedRow.update(curIndex, curField)
+ curIndex += 1
+ }
+ projectedRow
+ }
+
+ /**
+ * Projects provided instance of [[IndexedRecord]] into provided schema, assuming that the
+ * the schema of the original row is strictly a superset of the given one
+ */
+ def projectAvroUnsafe(record: IndexedRecord,
+ projectedSchema: Schema,
+ ordinals: List[Int],
+ recordBuilder: GenericRecordBuilder): GenericRecord = {
+ val fields = projectedSchema.getFields.asScala
+ assert(fields.length == ordinals.length)
Review comment:
use `checkstate` with a message ?
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -108,270 +136,284 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
}
}
- private def logFileIterator(split: HoodieMergeOnReadFileSplit,
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- override def hasNext: Boolean = {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], iterates over all of the records stored in
+ * Delta Log files (represented as [[InternalRow]]s)
+ */
+ private class LogFileIterator(split: HoodieMergeOnReadFileSplit,
+ config: Configuration)
+ extends Iterator[InternalRow] with Closeable with AvroDeserializerSupport {
+
+ protected override val requiredAvroSchema: Schema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
+ protected override val requiredStructTypeSchema: StructType = requiredSchema.structTypeSchema
+
+ protected val logFileReaderAvroSchema: Schema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
+
+ protected val recordBuilder: GenericRecordBuilder = new GenericRecordBuilder(requiredAvroSchema)
+ protected var recordToLoad: InternalRow = _
+
+ // TODO validate whether we need to do UnsafeProjection
+ protected val unsafeProjection: UnsafeProjection = UnsafeProjection.create(requiredStructTypeSchema)
+
+ // NOTE: This maps _required_ schema fields onto the _full_ table schema, collecting their "ordinals"
+ // w/in the record payload. This is required, to project records read from the Delta Log file
+ // which always reads records in full schema (never projected, due to the fact that DL file might
+ // be stored in non-columnar formats like Avro, HFile, etc)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, logFileReaderAvroSchema)
+
+ private var logScanner =
+ HoodieMergeOnReadRDD.scanLog(split.logFiles, getPartitionPath(split), logFileReaderAvroSchema, tableState,
+ maxCompactionMemoryInBytes, config)
+
+ private val logRecords = logScanner.getRecords.asScala
+
+ // NOTE: This iterator iterates over already projected (in required schema) records
+ // NOTE: This have to stay lazy to make sure it's initialized only at the point where it's
+ // going to be used, since we modify `logRecords` before that and therefore can't do it any earlier
+ protected lazy val logRecordsIterator: Iterator[Option[GenericRecord]] =
+ logRecords.iterator.map {
+ case (_, record) =>
+ val avroRecordOpt = toScalaOption(record.getData.getInsertValue(logFileReaderAvroSchema, payloadProps))
+ avroRecordOpt.map {
+ avroRecord => projectAvroUnsafe(avroRecord, requiredAvroSchema, requiredSchemaFieldOrdinals, recordBuilder)
}
- } else {
- false
- }
}
- override def next(): InternalRow = {
- recordToLoad
- }
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
- }
+ protected def removeLogRecord(key: String): Option[HoodieRecord[_ <: HoodieRecordPayload[_]]] =
+ logRecords.remove(key)
- private def skipMergeFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val deserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- recordToLoad = unsafeProjection(curRow)
- true
+ override def hasNext: Boolean =
+ logRecordsIterator.hasNext && {
+ val avroRecordOpt = logRecordsIterator.next()
+ if (avroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curAvrokey = logRecordsKeyIterator.next()
- val curAvroRecord = logRecords.get(curAvrokey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!curAvroRecord.isPresent) {
- // delete record found, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(curAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = deserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- false
- }
+ recordToLoad = unsafeProjection(deserialize(avroRecordOpt.get))
+ true
}
}
- override def next(): InternalRow = {
- recordToLoad
- }
+ override final def next(): InternalRow = recordToLoad
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
+ override def close(): Unit =
+ if (logScanner != null) {
+ try {
+ logScanner.close()
+ } finally {
+ logScanner = null
}
}
+ }
+
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * Base file as well as all of the Delta Log files simply returning concatenation of these streams, while not
+ * performing any combination/merging of the records w/ the same primary keys (ie producing duplicates potentially)
+ */
+ private class SkipMergeIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRow = baseFileIterator.next()
+ recordToLoad = unsafeProjection(curRow)
+ true
+ } else {
+ super[LogFileIterator].hasNext
+ }
}
+ }
- private def payloadCombineFileIterator(split: HoodieMergeOnReadFileSplit,
- baseFileIterator: Iterator[InternalRow],
- config: Configuration): Iterator[InternalRow] =
- new Iterator[InternalRow] with Closeable with SparkAdapterSupport {
- private val tableAvroSchema = new Schema.Parser().parse(tableSchema.avroSchemaStr)
- private val requiredAvroSchema = new Schema.Parser().parse(requiredSchema.avroSchemaStr)
- private val requiredFieldPosition =
- requiredSchema.structTypeSchema
- .map(f => tableAvroSchema.getField(f.name).pos()).toList
- private val serializer = sparkAdapter.createAvroSerializer(tableSchema.structTypeSchema, tableAvroSchema,
- resolveAvroSchemaNullability(tableAvroSchema))
- private val requiredDeserializer = sparkAdapter.createAvroDeserializer(requiredAvroSchema, requiredSchema.structTypeSchema)
- private val recordBuilder = new GenericRecordBuilder(requiredAvroSchema)
- private val unsafeProjection = UnsafeProjection.create(requiredSchema.structTypeSchema)
- private var logScanner = HoodieMergeOnReadRDD.scanLog(split, tableAvroSchema, config)
- private val logRecords = logScanner.getRecords
- private val logRecordsKeyIterator = logRecords.keySet().iterator().asScala
- private val keyToSkip = mutable.Set.empty[String]
- private val recordKeyPosition = tableSchema.structTypeSchema.fieldIndex(recordKeyField)
-
- private var recordToLoad: InternalRow = _
-
- @scala.annotation.tailrec
- override def hasNext: Boolean = {
- if (baseFileIterator.hasNext) {
- val curRow = baseFileIterator.next()
- val curKey = curRow.getString(recordKeyPosition)
- if (logRecords.containsKey(curKey)) {
- // duplicate key found, merging
- keyToSkip.add(curKey)
- val mergedAvroRecord = mergeRowWithLog(curRow, curKey)
- if (!mergedAvroRecord.isPresent) {
- // deleted
- this.hasNext
- } else {
- // load merged record as InternalRow with required schema
- val requiredAvroRecord = AvroConversionUtils.buildAvroRecordBySchema(mergedAvroRecord.get(), requiredAvroSchema,
- requiredFieldPosition, recordBuilder)
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- } else {
- // No merge needed, load current row with required schema
- recordToLoad = unsafeProjection(createInternalRowWithSchema(curRow, requiredSchema.structTypeSchema, requiredFieldPosition))
- true
- }
+ /**
+ * Provided w/ instance of [[HoodieMergeOnReadFileSplit]], provides an iterator over all of the records stored in
+ * a) Base file and all of the b) Delta Log files combining records with the same primary key from both of these
+ * streams
+ */
+ private class RecordMergingFileIterator(split: HoodieMergeOnReadFileSplit,
+ baseFileIterator: Iterator[InternalRow],
+ baseFileReaderSchema: HoodieTableSchema,
+ config: Configuration)
+ extends LogFileIterator(split, config) {
+
+ // NOTE: Record-merging iterator supports 2 modes of operation merging records bearing either
+ // - Full table's schema
+ // - Projected schema
+ // As such, no particular schema could be assumed, and therefore we rely on the caller
+ // to correspondingly set the scheme of the expected output of base-file reader
+ private val baseFileReaderAvroSchema = new Schema.Parser().parse(baseFileReaderSchema.avroSchemaStr)
+ private val requiredSchemaFieldOrdinals: List[Int] = collectFieldOrdinals(requiredAvroSchema, baseFileReaderAvroSchema)
+
+ private val serializer = sparkAdapter.createAvroSerializer(baseFileReaderSchema.structTypeSchema,
+ baseFileReaderAvroSchema, resolveAvroSchemaNullability(baseFileReaderAvroSchema))
+
+ private val recordKeyOrdinal = baseFileReaderSchema.structTypeSchema.fieldIndex(tableState.recordKeyField)
+
+ override def hasNext: Boolean = {
+ if (baseFileIterator.hasNext) {
+ val curRowRecord = baseFileIterator.next()
+ val curKey = curRowRecord.getString(recordKeyOrdinal)
+ val updatedRecordOpt = removeLogRecord(curKey)
+ if (updatedRecordOpt.isEmpty) {
+ // No merge needed, load current row with required projected schema
+ recordToLoad = unsafeProjection(projectRowUnsafe(curRowRecord, requiredSchema.structTypeSchema, requiredSchemaFieldOrdinals))
+ true
} else {
- if (logRecordsKeyIterator.hasNext) {
- val curKey = logRecordsKeyIterator.next()
- if (keyToSkip.contains(curKey)) {
- this.hasNext
- } else {
- val insertAvroRecord = logRecords.get(curKey).getData.getInsertValue(tableAvroSchema, payloadProps)
- if (!insertAvroRecord.isPresent) {
- // stand alone delete record, skipping
- this.hasNext
- } else {
- val requiredAvroRecord = AvroConversionUtils
- .buildAvroRecordBySchema(
- insertAvroRecord.get(),
- requiredAvroSchema,
- requiredFieldPosition,
- recordBuilder
- )
- val rowOpt = requiredDeserializer.deserialize(requiredAvroRecord)
- recordToLoad = unsafeProjection(rowOpt.get.asInstanceOf[InternalRow])
- true
- }
- }
+ val mergedAvroRecordOpt = merge(serialize(curRowRecord), updatedRecordOpt.get)
+ if (mergedAvroRecordOpt.isEmpty) {
+ // Record has been deleted, skipping
+ this.hasNext
} else {
- false
+ // NOTE: In occurrence of a merge we can't know the schema of the record being returned, b/c
+ // record from the Delta Log will bear (full) Table schema, while record from the Base file
+ // might already be read in projected one (as an optimization).
+ // As such we can't use more performant [[projectAvroUnsafe]], and instead have to fallback
+ // to [[projectAvro]]
+ val projectedAvroRecord = projectAvro(mergedAvroRecordOpt.get, requiredAvroSchema, recordBuilder)
+ recordToLoad = unsafeProjection(deserialize(projectedAvroRecord))
+ true
}
}
+ } else {
+ super[LogFileIterator].hasNext
}
+ }
- override def next(): InternalRow = recordToLoad
-
- override def close(): Unit = {
- if (logScanner != null) {
- try {
- logScanner.close()
- } finally {
- logScanner = null
- }
- }
- }
+ private def serialize(curRowRecord: InternalRow): GenericRecord =
+ serializer.serialize(curRowRecord).asInstanceOf[GenericRecord]
- private def mergeRowWithLog(curRow: InternalRow, curKey: String) : org.apache.hudi.common.util.Option[IndexedRecord] = {
- val historyAvroRecord = serializer.serialize(curRow).asInstanceOf[GenericRecord]
- val mergedRec = logRecords.get(curKey).getData
- .combineAndGetUpdateValue(historyAvroRecord, tableAvroSchema, payloadProps)
- if (mergedRec.isPresent && mergedRec.get().getSchema != tableAvroSchema) {
- org.apache.hudi.common.util.Option.of(HoodieAvroUtils.rewriteRecord(mergedRec.get().asInstanceOf[GenericRecord], tableAvroSchema).asInstanceOf[IndexedRecord])
- } else {
- mergedRec
- }
- }
+ private def merge(curAvroRecord: GenericRecord, newRecord: HoodieRecord[_ <: HoodieRecordPayload[_]]): Option[IndexedRecord] = {
+ // NOTE: We have to pass in Avro Schema used to read from Delta Log file since we invoke combining API
+ // on the record from the Delta Log
+ toScalaOption(newRecord.getData.combineAndGetUpdateValue(curAvroRecord, logFileReaderAvroSchema, payloadProps))
}
+ }
}
private object HoodieMergeOnReadRDD {
+
val CONFIG_INSTANTIATION_LOCK = new Object()
- def scanLog(split: HoodieMergeOnReadFileSplit, logSchema: Schema, config: Configuration): HoodieMergedLogRecordScanner = {
- val fs = FSUtils.getFs(split.tablePath, config)
- val logFiles = split.logFiles.get
+ def scanLog(logFiles: List[HoodieLogFile],
+ partitionPath: Path,
+ logSchema: Schema,
+ tableState: HoodieTableState,
+ maxCompactionMemoryInBytes: Long,
+ hadoopConf: Configuration): HoodieMergedLogRecordScanner = {
Review comment:
this comment seems not addressed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078558718
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064748224
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064831876
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b07cca5112163e153385c690203603b74542ace6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1065839129
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b8272f7316953648a46fafe5065965a4552a7e0e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068541489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
* c6b213eac29626963bf8bcb998ec96c700c6ae82 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1068541489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
* c6b213eac29626963bf8bcb998ec96c700c6ae82 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067396258
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1069513264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c6b213eac29626963bf8bcb998ec96c700c6ae82 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979)
* c7321c52d6f6d883e40d7331c782d0ea265172b6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071516654
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c7321c52d6f6d883e40d7331c782d0ea265172b6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006)
* 1f7cca17fffc7f559d5f4580f711efbee5e2b55f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072004489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074743538
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074835414
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r832629643
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +73,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
+
case logFileOnlySplit if logFileOnlySplit.dataFile.isEmpty =>
- logFileIterator(logFileOnlySplit, getConfig)
- case skipMergeSplit if skipMergeSplit.mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
- skipMergeFileIterator(skipMergeSplit, requiredSchemaFileReader(skipMergeSplit.dataFile.get), getConfig)
- case payloadCombineSplit
- if payloadCombineSplit.mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
- payloadCombineFileIterator(payloadCombineSplit, fullSchemaFileReader(payloadCombineSplit.dataFile.get),
- getConfig)
+ new LogFileIterator(logFileOnlySplit, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
+ val baseFileIterator = requiredSchemaFileReader.apply(split.dataFile.get)
+ new SkipMergeIterator(split, baseFileIterator, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
+ val (baseFileIterator, schema) = readBaseFile(split)
+ new RecordMergingFileIterator(split, baseFileIterator, schema, getConfig)
+
case _ => throw new HoodieException(s"Unable to select an Iterator to read the Hoodie MOR File Split for " +
s"file path: ${mergeOnReadPartition.split.dataFile.get.filePath}" +
s"log paths: ${mergeOnReadPartition.split.logFiles.toString}" +
- s"hoodie table path: ${mergeOnReadPartition.split.tablePath}" +
+ s"hoodie table path: ${tableState.tablePath}" +
s"spark partition Index: ${mergeOnReadPartition.index}" +
- s"merge type: ${mergeOnReadPartition.split.mergeType}")
+ s"merge type: ${mergeType}")
}
+
if (iter.isInstanceOf[Closeable]) {
// register a callback to close logScanner which will be executed on task completion.
// when tasks finished, this method will be called, and release resources.
Option(TaskContext.get()).foreach(_.addTaskCompletionListener[Unit](_ => iter.asInstanceOf[Closeable].close()))
}
+
iter
}
+ private def readBaseFile(split: HoodieMergeOnReadFileSplit): (Iterator[InternalRow], HoodieTableSchema) = {
+ // NOTE: This is an optimization making sure that even for MOR tables we fetch absolute minimum
+ // of the stored data possible, while still properly executing corresponding relation's semantic
+ // and meet the query's requirements.
+ //
+ // Here we assume that iff queried table
+ // a) It does use one of the standard (and whitelisted) Record Payload classes
+ // then we can avoid reading and parsing the records w/ _full_ schema, and instead only
+ // rely on projected one, nevertheless being able to perform merging correctly
+ if (!whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
+ (fullSchemaFileReader(split.dataFile.get), tableSchema)
Review comment:
It actually was `fullSchema` at some point but this still left room for confusion in regards to what this full schema might refer to, so i've renamed it to `tableSchema`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075643139
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* a12b35f76e18d194d41b36a47f02d3b96e9f01e3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165)
* 366ec612b7a31fe375730161f470a9135a7d3ff9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074693009
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1049082728
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071819401
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050387755
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072882599
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072054380
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b74447d7df84648415dcc5c3b8d7e01bf6123b8c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071819401
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59ff930ff76d233229c9384215ff4158a81eecd5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1071514052
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067335814
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067333896
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1072901323
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074416349
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074693009
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* d8c47c90b2b494ca543a36b130d853be0bbc6ba4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1075690787
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* 366ec612b7a31fe375730161f470a9135a7d3ff9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074547340
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074510628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074656503
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1078695230
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
}, {
"hash" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fbe3701ae273bc308c0141223155316c0c3087b5",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7149",
"triggerID" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"triggerType" : "PUSH"
}, {
"hash" : "d8c47c90b2b494ca543a36b130d853be0bbc6ba4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7155",
"triggerID" : "1074656503",
"triggerType" : "MANUAL"
}, {
"hash" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7165",
"triggerID" : "a12b35f76e18d194d41b36a47f02d3b96e9f01e3",
"triggerType" : "PUSH"
}, {
"hash" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7200",
"triggerID" : "366ec612b7a31fe375730161f470a9135a7d3ff9",
"triggerType" : "PUSH"
}, {
"hash" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327",
"triggerID" : "c68818933f0a440f928feeccf33e5704fe0a165c",
"triggerType" : "PUSH"
}, {
"hash" : "7bfcb40af344c2589311f0565384387a172f9304",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354",
"triggerID" : "7bfcb40af344c2589311f0565384387a172f9304",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* fbe3701ae273bc308c0141223155316c0c3087b5 UNKNOWN
* c68818933f0a440f928feeccf33e5704fe0a165c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7327)
* 7bfcb40af344c2589311f0565384387a172f9304 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7354)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan merged pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan merged pull request #4888:
URL: https://github.com/apache/hudi/pull/4888
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064723457
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1064748224
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e0afa9f1de90411220a6c1d25c0c9e43f09f6baf Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815)
* b07cca5112163e153385c690203603b74542ace6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050359771
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050322824
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050324902
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282)
* a26fcbe05b549c51a141b112119417b3717036d2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1050320927
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 425a84550ef93abd600fa334a7526c2364f0c229 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245)
* dbc0ceb813668e01e0c2aec178aa9b12034c3382 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396][Stacked on 4877] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1067333896
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599584e29b5fbafa702815b8a441ecaef2ed6009 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882)
* 1bec7a15ab65e0f241ec4ebef5da2942e5178ca7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r835435015
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieMergeOnReadRDD.scala
##########
@@ -70,34 +76,59 @@ class HoodieMergeOnReadRDD(@transient sc: SparkContext,
)
.getOrElse(new Properties())
+ private val whitelistedPayloadClasses: Set[String] = Seq(
+ classOf[OverwriteWithLatestAvroPayload]
+ ).map(_.getName).toSet
+
override def compute(split: Partition, context: TaskContext): Iterator[InternalRow] = {
val mergeOnReadPartition = split.asInstanceOf[HoodieMergeOnReadPartition]
val iter = mergeOnReadPartition.split match {
case dataFileOnlySplit if dataFileOnlySplit.logFiles.isEmpty =>
- requiredSchemaFileReader(dataFileOnlySplit.dataFile.get)
+ requiredSchemaFileReader.apply(dataFileOnlySplit.dataFile.get)
+
case logFileOnlySplit if logFileOnlySplit.dataFile.isEmpty =>
- logFileIterator(logFileOnlySplit, getConfig)
- case skipMergeSplit if skipMergeSplit.mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
- skipMergeFileIterator(skipMergeSplit, requiredSchemaFileReader(skipMergeSplit.dataFile.get), getConfig)
- case payloadCombineSplit
- if payloadCombineSplit.mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
- payloadCombineFileIterator(payloadCombineSplit, fullSchemaFileReader(payloadCombineSplit.dataFile.get),
- getConfig)
+ new LogFileIterator(logFileOnlySplit, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_SKIP_MERGE_OPT_VAL) =>
+ val baseFileIterator = requiredSchemaFileReader.apply(split.dataFile.get)
+ new SkipMergeIterator(split, baseFileIterator, getConfig)
+
+ case split if mergeType.equals(DataSourceReadOptions.REALTIME_PAYLOAD_COMBINE_OPT_VAL) =>
+ val (baseFileIterator, schema) = readBaseFile(split)
+ new RecordMergingFileIterator(split, baseFileIterator, schema, getConfig)
+
case _ => throw new HoodieException(s"Unable to select an Iterator to read the Hoodie MOR File Split for " +
s"file path: ${mergeOnReadPartition.split.dataFile.get.filePath}" +
s"log paths: ${mergeOnReadPartition.split.logFiles.toString}" +
- s"hoodie table path: ${mergeOnReadPartition.split.tablePath}" +
+ s"hoodie table path: ${tableState.tablePath}" +
s"spark partition Index: ${mergeOnReadPartition.index}" +
- s"merge type: ${mergeOnReadPartition.split.mergeType}")
+ s"merge type: ${mergeType}")
}
+
if (iter.isInstanceOf[Closeable]) {
// register a callback to close logScanner which will be executed on task completion.
// when tasks finished, this method will be called, and release resources.
Option(TaskContext.get()).foreach(_.addTaskCompletionListener[Unit](_ => iter.asInstanceOf[Closeable].close()))
}
+
iter
}
+ private def readBaseFile(split: HoodieMergeOnReadFileSplit): (Iterator[InternalRow], HoodieTableSchema) = {
+ // NOTE: This is an optimization making sure that even for MOR tables we fetch absolute minimum
+ // of the stored data possible, while still properly executing corresponding relation's semantic
+ // and meet the query's requirements.
+ //
+ // Here we assume that iff queried table
+ // a) It does use one of the standard (and whitelisted) Record Payload classes
+ // then we can avoid reading and parsing the records w/ _full_ schema, and instead only
+ // rely on projected one, nevertheless being able to perform merging correctly
+ if (!whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
Review comment:
```suggestion
if (whitelistedPayloadClasses.contains(tableState.recordPayloadClassName))
```
ok i just meant without negation it reads better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074371687
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#discussion_r831616177
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -54,8 +55,12 @@ trait HoodieFileSplit {}
case class HoodieTableSchema(structTypeSchema: StructType, avroSchemaStr: String)
-case class HoodieTableState(recordKeyField: String,
- preCombineFieldOpt: Option[String])
+case class HoodieTableState(tablePath: String,
+ latestCommit: String,
Review comment:
latestCommitTime
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BaseFileOnlyRelation.scala
##########
@@ -52,6 +53,14 @@ class BaseFileOnlyRelation(sqlContext: SQLContext,
override type FileSplit = HoodieBaseFileSplit
+ override lazy val mandatoryColumns: Seq[String] = {
+ if (isMetadataTable(metaClient)) {
Review comment:
lets remove this special casing for metadata table? It's dealing with an abstraction few layers deeper than here.
##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala
##########
@@ -79,13 +84,30 @@ abstract class HoodieBaseRelation(val sqlContext: SQLContext,
protected lazy val basePath: String = metaClient.getBasePath
- // If meta fields are enabled, always prefer key from the meta field as opposed to user-specified one
- // NOTE: This is historical behavior which is preserved as is
+ // NOTE: Record key-field is assumed singular here due to the either of
Review comment:
same. can we avoid references to metadata table from this layer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074365979
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f326e751ecf0f9aa93e0d94271b4154b3f8e9479 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080)
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074445715
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074445715
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4888: [HUDI-3396] Refactoring `MergeOnReadRDD` to avoid duplication, fetch only projected columns
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4888:
URL: https://github.com/apache/hudi/pull/4888#issuecomment-1074510628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "425a84550ef93abd600fa334a7526c2364f0c229",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6245",
"triggerID" : "425a84550ef93abd600fa334a7526c2364f0c229",
"triggerType" : "PUSH"
}, {
"hash" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6282",
"triggerID" : "dbc0ceb813668e01e0c2aec178aa9b12034c3382",
"triggerType" : "PUSH"
}, {
"hash" : "a26fcbe05b549c51a141b112119417b3717036d2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6285",
"triggerID" : "a26fcbe05b549c51a141b112119417b3717036d2",
"triggerType" : "PUSH"
}, {
"hash" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6815",
"triggerID" : "e0afa9f1de90411220a6c1d25c0c9e43f09f6baf",
"triggerType" : "PUSH"
}, {
"hash" : "b07cca5112163e153385c690203603b74542ace6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6820",
"triggerID" : "b07cca5112163e153385c690203603b74542ace6",
"triggerType" : "PUSH"
}, {
"hash" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6870",
"triggerID" : "b8272f7316953648a46fafe5065965a4552a7e0e",
"triggerType" : "PUSH"
}, {
"hash" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6882",
"triggerID" : "599584e29b5fbafa702815b8a441ecaef2ed6009",
"triggerType" : "PUSH"
}, {
"hash" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6937",
"triggerID" : "1bec7a15ab65e0f241ec4ebef5da2942e5178ca7",
"triggerType" : "PUSH"
}, {
"hash" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6979",
"triggerID" : "c6b213eac29626963bf8bcb998ec96c700c6ae82",
"triggerType" : "PUSH"
}, {
"hash" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7006",
"triggerID" : "c7321c52d6f6d883e40d7331c782d0ea265172b6",
"triggerType" : "PUSH"
}, {
"hash" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7044",
"triggerID" : "1f7cca17fffc7f559d5f4580f711efbee5e2b55f",
"triggerType" : "PUSH"
}, {
"hash" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7046",
"triggerID" : "59ff930ff76d233229c9384215ff4158a81eecd5",
"triggerType" : "PUSH"
}, {
"hash" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7062",
"triggerID" : "b74447d7df84648415dcc5c3b8d7e01bf6123b8c",
"triggerType" : "PUSH"
}, {
"hash" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7080",
"triggerID" : "f326e751ecf0f9aa93e0d94271b4154b3f8e9479",
"triggerType" : "PUSH"
}, {
"hash" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de2c88160bb4f8dfa49855ad0236df0ca9e947c1",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140",
"triggerID" : "484ab4138dba113df585fd3fb011607e0715f928",
"triggerType" : "PUSH"
}, {
"hash" : "484ab4138dba113df585fd3fb011607e0715f928",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141",
"triggerID" : "1074444305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* de2c88160bb4f8dfa49855ad0236df0ca9e947c1 UNKNOWN
* 484ab4138dba113df585fd3fb011607e0715f928 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7140) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7141)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org