You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/02/17 10:31:57 UTC
[GitHub] [hudi] cuibo01 opened a new pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
cuibo01 opened a new pull request #4837:
URL: https://github.com/apache/hudi/pull/4837
…dRecords
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045592192
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042909383
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045610254
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1046001527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
}, {
"hash" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144",
"triggerID" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
* 9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042885672
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042885672
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045613884
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042861087
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042806597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042842723
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1046001527
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
}, {
"hash" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144",
"triggerID" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
* 9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045588535
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042822441
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045588535
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045682060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045682060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045993947
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a change in pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#discussion_r809779408
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -134,17 +162,30 @@
@Override
public List<GenericRecord> readAvroRecords(Configuration configuration, Path filePath, Schema avroSchema) {
List<GenericRecord> records = new ArrayList<>();
+ Reader reader = null;
+ RecordReader recordReader = null;
try {
Review comment:
they can't be final.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042883088
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045606581
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1043002695
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042883088
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045661058
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042803968
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042801617
hi @danny0405, pls review the pr :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a change in pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#discussion_r809779408
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -134,17 +162,30 @@
@Override
public List<GenericRecord> readAvroRecords(Configuration configuration, Path filePath, Schema avroSchema) {
List<GenericRecord> records = new ArrayList<>();
+ Reader reader = null;
+ RecordReader recordReader = null;
try {
Review comment:
they can't be final， because they need to be close
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042822441
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
danny0405 commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045965280
Did you notice that i have attached a patch for this pr? Please apply it then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045584987
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045784891
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045613884
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045936874
> [HUDI-3446.patch.zip](https://github.com/apache/hudi/files/8101564/HUDI-3446.patch.zip) Thanks, I have created a path for you to save review time, please validate the logic the test the correctness locally.
The PR is correct by TestParquetUtils#testFetchRecordKeyPartitionPathFromParquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042806597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a change in pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
danny0405 commented on a change in pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#discussion_r809692854
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/BaseFileUtils.java
##########
@@ -171,6 +172,33 @@ public BloomFilter readBloomFilterFromMetadata(Configuration configuration, Path
*/
public abstract List<HoodieKey> fetchRecordKeyPartitionPath(Configuration configuration, Path filePath);
+ /**
+ * Fetch {@link HoodieKey}s from the given data file.
+ * @param reader The file reader
+ * @param filePath The data file path
+ * @return {@link List} of {@link HoodieKey}s fetched from the parquet file
+ */
+ public abstract List<HoodieKey> fetchRecordKeyPartitionPath(BaseFileReader reader, Path filePath, int batchSize);
+
+ /**
+ * Open File Reader.
+ * @param configuration configuration to build file reader
+ * @param filePath The data file path
+ * @param keyGeneratorOpt instance of KeyGenerator
+ * @return file reader
+ */
+ public abstract BaseFileReader getReader(Configuration configuration, Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt);
+
+ /**
+ * Open File Reader.
+ * @param configuration configuration to build file reader
+ * @param filePath The data file path
+ * @return file reader
+ */
+ public BaseFileReader getReader(Configuration configuration, Path filePath) {
+ return getReader(configuration, filePath, Option.empty());
Review comment:
ditto
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -134,17 +162,30 @@
@Override
public List<GenericRecord> readAvroRecords(Configuration configuration, Path filePath, Schema avroSchema) {
List<GenericRecord> records = new ArrayList<>();
+ Reader reader = null;
+ RecordReader recordReader = null;
try {
Review comment:
```java
final Reader reader = null;
final RecordReader recordReader = null;
```
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/BaseFileUtils.java
##########
@@ -171,6 +172,33 @@ public BloomFilter readBloomFilterFromMetadata(Configuration configuration, Path
*/
public abstract List<HoodieKey> fetchRecordKeyPartitionPath(Configuration configuration, Path filePath);
+ /**
+ * Fetch {@link HoodieKey}s from the given data file.
+ * @param reader The file reader
+ * @param filePath The data file path
+ * @return {@link List} of {@link HoodieKey}s fetched from the parquet file
+ */
+ public abstract List<HoodieKey> fetchRecordKeyPartitionPath(BaseFileReader reader, Path filePath, int batchSize);
+
+ /**
+ * Open File Reader.
+ * @param configuration configuration to build file reader
+ * @param filePath The data file path
+ * @param keyGeneratorOpt instance of KeyGenerator
+ * @return file reader
+ */
+ public abstract BaseFileReader getReader(Configuration configuration, Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt);
+
Review comment:
`getReader` => `getRecordKeyPartitionPathReader`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -239,11 +291,30 @@ public Schema readAvroSchema(Configuration conf, Path orcFilePath) {
@Override
public long getRowCount(Configuration conf, Path orcFilePath) {
- try {
- Reader reader = OrcFile.createReader(orcFilePath, OrcFile.readerOptions(conf));
+ try (Reader reader = OrcFile.createReader(orcFilePath, OrcFile.readerOptions(conf))) {
return reader.getNumberOfRows();
} catch (IOException io) {
throw new HoodieIOException("Unable to get row count for ORC file:" + orcFilePath, io);
}
}
+
+ private class OrcFileInnerReader extends BaseFileReader {
+ Reader reader;
Review comment:
`OrcFileInnerReader` => `OrcReaderWrapper`
##########
File path: hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapOperator.java
##########
@@ -211,16 +213,29 @@ protected void loadRecords(String partitionPath) throws Exception {
return;
}
- final List<HoodieKey> hoodieKeys;
- try {
- hoodieKeys =
- fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new Path(baseFile.getPath()));
- } catch (Exception e) {
- throw new HoodieException(String.format("Error when loading record keys from file: %s", baseFile), e);
- }
-
- for (HoodieKey hoodieKey : hoodieKeys) {
- output.collect(new StreamRecord(new IndexRecord(generateHoodieRecord(hoodieKey, fileSlice))));
+ if (this.batchSize > 0) {
+ List<HoodieKey> hoodieKeys;
+ Path filePath = new Path(baseFile.getPath());
+ try (BaseFileUtils.BaseFileReader reader = fileUtils.getReader(this.hadoopConf, filePath)) {
+ do {
+ hoodieKeys = fileUtils.fetchRecordKeyPartitionPath(reader, filePath, batchSize);
Review comment:
1. Can we have method `BaseFileReader#hasNextBatch` and `BaseFileReader#NextBatch` then iterator over that ?
2. Can we also have a constant batch size `1024` and remove the config option then ?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -100,6 +126,9 @@
String rowKey = rowKeys.toString(i);
String partitionPath = partitionPaths.toString(i);
hoodieKeys.add(new HoodieKey(rowKey, partitionPath));
+ if (batchSize > 0 && hoodieKeys.size() >= batchSize) {
+ break;
+ }
Review comment:
1. I see that there is already a batchSize param in `VectorizedRowBatch` and the default val is 1024, can we use that
2. We should not skip the for-loop early in while reading a batch, which would cause data lost, instead we can put it out of the for-loop, that means we must read a full `VectorizedRowBatch`
3. Considering there is already a batch there, the more reasonable way to control the read buffer is a param like `int batches`, which means the number of read batches once a time. Or you can divide the `batchSize` by the `VectorizedRowBatch` size to get the vector batch number to read.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java
##########
@@ -143,11 +148,20 @@
fields.addAll(keyGenerator.getPartitionPathFields());
return HoodieAvroUtils.getSchemaForFields(readAvroSchema(conf, filePath), fields);
})
- .orElse(HoodieAvroUtils.getRecordKeyPartitionPathSchema());
+ .orElse(HoodieAvroUtils.getRecordKeyPartitionPathSchema());
AvroReadSupport.setAvroReadSchema(conf, readSchema);
Review comment:
Fix the indentation.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/BaseFileUtils.java
##########
@@ -171,6 +172,33 @@ public BloomFilter readBloomFilterFromMetadata(Configuration configuration, Path
*/
public abstract List<HoodieKey> fetchRecordKeyPartitionPath(Configuration configuration, Path filePath);
+ /**
+ * Fetch {@link HoodieKey}s from the given data file.
+ * @param reader The file reader
+ * @param filePath The data file path
+ * @return {@link List} of {@link HoodieKey}s fetched from the parquet file
Review comment:
1. Please align the comments of the param document, please fix all the documents
2. add document for `batchSize`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/ParquetUtils.java
##########
@@ -424,4 +435,17 @@ private static BigDecimal extractDecimal(Object val, DecimalMetadata decimalMeta
throw new UnsupportedOperationException(String.format("Unsupported value type (%s)", val.getClass().getName()));
}
}
+
+ private class ParquetFileInnerReader extends BaseFileReader {
+ ParquetReader reader;
Review comment:
`ParquetFileInnerReader` => `ParquetReaderWrapper`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045592192
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1046012623
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
}, {
"hash" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144",
"triggerID" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
* 9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6144)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
danny0405 merged pull request #4837:
URL: https://github.com/apache/hudi/pull/4837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045993947
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042861087
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045661058
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
danny0405 commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045874545
[HUDI-3446.patch.zip](https://github.com/apache/hudi/files/8101564/HUDI-3446.patch.zip)
Thanks, I have created a path for you to save review time, please validate the logic the test the correctness locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042803968
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042825135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042842723
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1043002695
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042909383
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 7f14898f3d72ac64754b70e2af7e71955122370c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090)
* 38e390e15df681761491177da56e5f032a940d49 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1042825135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 86ee140c8551395f3c33fb39444add4aaeca9a2e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088)
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1044136388
thx for your review @danny0405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045996132
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
}, {
"hash" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
* 9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045996132
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
}, {
"hash" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ab83de93be57f401f8e503b2f9a22f4597edf1e",
"triggerType" : "PUSH"
}, {
"hash" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
* 1ab83de93be57f401f8e503b2f9a22f4597edf1e UNKNOWN
* 9c8cd84551964d1000a0ca6c2a6ad9f59aa52afa UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045610254
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140)
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045606581
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 2ca7c188ece5e24edd31986efd21bb99e99952dc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045584987
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* 38e390e15df681761491177da56e5f032a940d49 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096)
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a change in pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#discussion_r810435202
##########
File path: hudi-flink/src/main/java/org/apache/hudi/sink/bootstrap/BootstrapOperator.java
##########
@@ -211,16 +213,29 @@ protected void loadRecords(String partitionPath) throws Exception {
return;
}
- final List<HoodieKey> hoodieKeys;
- try {
- hoodieKeys =
- fileUtils.fetchRecordKeyPartitionPath(this.hadoopConf, new Path(baseFile.getPath()));
- } catch (Exception e) {
- throw new HoodieException(String.format("Error when loading record keys from file: %s", baseFile), e);
- }
-
- for (HoodieKey hoodieKey : hoodieKeys) {
- output.collect(new StreamRecord(new IndexRecord(generateHoodieRecord(hoodieKey, fileSlice))));
+ if (this.batchSize > 0) {
+ List<HoodieKey> hoodieKeys;
+ Path filePath = new Path(baseFile.getPath());
+ try (BaseFileUtils.BaseFileReader reader = fileUtils.getReader(this.hadoopConf, filePath)) {
+ do {
+ hoodieKeys = fileUtils.fetchRecordKeyPartitionPath(reader, filePath, batchSize);
Review comment:
Modify to ClosableIterator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a change in pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on a change in pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#discussion_r810435131
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java
##########
@@ -100,6 +126,9 @@
String rowKey = rowKeys.toString(i);
String partitionPath = partitionPaths.toString(i);
hoodieKeys.add(new HoodieKey(rowKey, partitionPath));
+ if (batchSize > 0 && hoodieKeys.size() >= batchSize) {
+ break;
+ }
Review comment:
Modify to closeableIterator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4837: [HUDI-3446][HUDI-FLINK] support batch Reader in BootstrapOperator#loadRecords
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4837:
URL: https://github.com/apache/hudi/pull/4837#issuecomment-1045784891
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6088",
"triggerID" : "86ee140c8551395f3c33fb39444add4aaeca9a2e",
"triggerType" : "PUSH"
}, {
"hash" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2",
"triggerType" : "PUSH"
}, {
"hash" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6090",
"triggerID" : "7f14898f3d72ac64754b70e2af7e71955122370c",
"triggerType" : "PUSH"
}, {
"hash" : "38e390e15df681761491177da56e5f032a940d49",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6096",
"triggerID" : "38e390e15df681761491177da56e5f032a940d49",
"triggerType" : "PUSH"
}, {
"hash" : "d5879634130634355953842cd8505f05424aae76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d5879634130634355953842cd8505f05424aae76",
"triggerType" : "PUSH"
}, {
"hash" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5170af298807cd290a9ba8c1121b4c25916c1814",
"triggerType" : "PUSH"
}, {
"hash" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2131d4e2d0e664fbe08030e95dd48cd7ed2707df",
"triggerType" : "PUSH"
}, {
"hash" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6140",
"triggerID" : "2ca7c188ece5e24edd31986efd21bb99e99952dc",
"triggerType" : "PUSH"
}, {
"hash" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "868fa6175b63d35db2bd4ecb7280732d316bb74b",
"triggerType" : "PUSH"
}, {
"hash" : "3ca0191014d2714fd63319aab36431236bd156ad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143",
"triggerID" : "3ca0191014d2714fd63319aab36431236bd156ad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ad7ea77c1eb29b28338ee84eb39528bfdf78f2 UNKNOWN
* d5879634130634355953842cd8505f05424aae76 UNKNOWN
* 5170af298807cd290a9ba8c1121b4c25916c1814 UNKNOWN
* 2131d4e2d0e664fbe08030e95dd48cd7ed2707df UNKNOWN
* 868fa6175b63d35db2bd4ecb7280732d316bb74b UNKNOWN
* 3ca0191014d2714fd63319aab36431236bd156ad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=6143)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org