You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/12/16 00:44:33 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request #4333: Adding support for Parquet in MOR `LogBlock`s
alexeykudinkin opened a new pull request #4333:
URL: https://github.com/apache/hudi/pull/4333
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
Adding support for Parquet in MOR tables Log blocks
NOTE: This change builds off progress of https://github.com/apache/hudi/pull/3930
## Brief change log
- Adding support for Parquet in MOR tables Log blocks
## Verify this pull request
[WIP] This change added tests and can be verified as follows:
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995426398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
* 1a390d804e170c0fd42014211db65fd44a3d054e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012686549
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r784457956
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java
##########
@@ -497,4 +489,32 @@ private void flushToDiskIfRequired(HoodieRecord record) {
numberOfRecords = 0;
}
}
+
+ protected void appendDataAndDeleteBlocks(Map<HeaderMetadataType, String> header) {
Review comment:
Frankly, can't understand why this method got moved. Nothing should have changed here. Let me revert this
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java
##########
@@ -497,4 +489,32 @@ private void flushToDiskIfRequired(HoodieRecord record) {
numberOfRecords = 0;
}
}
+
+ protected void appendDataAndDeleteBlocks(Map<HeaderMetadataType, String> header) {
Review comment:
Oh, i've found the root-cause of those inexplicable changes -- i guess after upgrade of IDEA i've got new box "Rearrange code" ticked automatically (never ticked it myself) and i've confirmed it being the root-cause of those random re-orderings
<img width="357" alt="Screen Shot 2022-01-13 at 6 10 49 PM" src="https://user-images.githubusercontent.com/428277/149439483-0d67b659-a61c-4ffe-807e-70a07e9d7d0e.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012785174
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027384679
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027651306
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
}, {
"hash" : "29dd8404768aa1811844efc1332d02041971b8e0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29dd8404768aa1811844efc1332d02041971b8e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
* 29dd8404768aa1811844efc1332d02041971b8e0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r796909001
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/model/HoodieLogFile.java
##########
@@ -60,7 +59,7 @@ public HoodieLogFile(FileStatus fileStatus) {
public HoodieLogFile(Path logPath) {
this.fileStatus = null;
this.pathStr = logPath.toString();
- this.fileLen = 0;
+ this.fileLen = -1;
Review comment:
So the file len of 0 is very confusing it could be a legitimate case. I had to go and check whether file was indeed of length 0.
Setting it to -1 makes it clear that this is not initialized and can't be used as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995351561
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995404984
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995404984
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996226567
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997129628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
* cecde3b6734576c5f2863ec2b4b90689600cb746 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998370268
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r778523226
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -88,76 +92,24 @@ public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSc
}
public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSchema, int bufferSize,
- boolean readBlockLazily, boolean reverseReader, boolean enableInlineReading,
+ boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups,
String keyField) throws IOException {
- FSDataInputStream fsDataInputStream = fs.open(logFile.getPath(), bufferSize);
- this.logFile = logFile;
- this.inputStream = getFSDataInputStream(fsDataInputStream, fs, bufferSize);
+ // NOTE: We repackage {@code HoodieLogFile} here to make sure that the provided path
+ // is prefixed with an appropriate scheme given that we're not propagating the FS
+ // further
+ this.logFile = new HoodieLogFile(fs.getFileStatus(logFile.getPath()));
Review comment:
👍
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/ByteBufferBackedInputFile.java
##########
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hudi.common.util.io.ByteBufferBackedInputStream;
+import org.apache.parquet.io.DelegatingSeekableInputStream;
+import org.apache.parquet.io.InputFile;
+import org.apache.parquet.io.SeekableInputStream;
+
+/**
+ * Implementation of {@link InputFile} backed by {@code byte[]} buffer
+ */
+public class ByteBufferBackedInputFile implements InputFile {
+ private final byte[] buffer;
+ private final int offset;
+ private final int length;
+
+ public ByteBufferBackedInputFile(byte[] buffer, int offset, int length) {
+ this.buffer = buffer;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public ByteBufferBackedInputFile(byte[] buffer) {
+ this(buffer, 0, buffer.length);
+ }
+
+ @Override
+ public long getLength() {
+ return length;
+ }
+
+ @Override
+ public SeekableInputStream newStream() {
Review comment:
These classes are a simple wrappers (around `ByteBufferBackednputStream` which bears UT) and are tested by FTs (they're used in DataBlocks). Do we still want to test them w/ standalone UTs?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -72,7 +76,7 @@
private long reverseLogFilePosition;
private long lastReverseLogFilePosition;
private boolean reverseReader;
- private boolean enableInlineReading;
+ private boolean enableRecordLookups;
Review comment:
Not sure i understand your point fully -- this flag in particular is used to gate whether we allow point lookups for records when `getRecords(<keys>)` method of the Block is used
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -148,65 +165,40 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected void createRecordsFromContentBytes() throws IOException {
- if (enableInlineReading) {
- getRecords(Collections.emptyList());
- } else {
- super.createRecordsFromContentBytes();
- }
- }
-
- @Override
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- readWithInlineFS(keys);
- return records;
- }
-
- private void readWithInlineFS(List<String> keys) throws IOException {
- boolean enableFullScan = keys.isEmpty();
- // Get schema from the header
- Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- // If readerSchema was not present, use writerSchema
- if (schema == null) {
- schema = writerSchema;
- }
- Configuration conf = new Configuration();
- CacheConfig cacheConf = new CacheConfig(conf);
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
Configuration inlineConf = new Configuration();
inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
Path inlinePath = InLineFSUtils.getInlineFilePath(
- getBlockContentLocation().get().getLogFile().getPath(),
- getBlockContentLocation().get().getLogFile().getPath().getFileSystem(conf).getScheme(),
- getBlockContentLocation().get().getContentPositionInLogFile(),
- getBlockContentLocation().get().getBlockSize());
- if (!enableFullScan) {
- // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
- Collections.sort(keys);
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
+ Collections.sort(keys);
+
+ try (HoodieHFileReader<IndexedRecord> reader =
+ new HoodieHFileReader<>(inlineConf, inlinePath, new CacheConfig(inlineConf), inlinePath.getFileSystem(inlineConf))) {
+ // Get writer's schema from the header
+ List<Pair<String, IndexedRecord>> logRecords = reader.readRecords(keys, readerSchema);
+ return logRecords.stream().map(Pair::getSecond).collect(Collectors.toList());
}
- HoodieHFileReader reader = new HoodieHFileReader(inlineConf, inlinePath, cacheConf, inlinePath.getFileSystem(inlineConf));
- List<org.apache.hadoop.hbase.util.Pair<String, IndexedRecord>> logRecords = enableFullScan ? reader.readAllRecords(writerSchema, schema) :
- reader.readRecords(keys, schema);
- reader.close();
- this.records = logRecords.stream().map(t -> t.getSecond()).collect(Collectors.toList());
}
@Override
- protected void deserializeRecords() throws IOException {
+ protected List<IndexedRecord> deserializeRecords(byte[] content) throws IOException {
+ checkState(readerSchema != null, "Reader's schema has to be non-null");
Review comment:
Correct. Now it'd never be null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012717317
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r791213691
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##########
@@ -719,19 +717,6 @@ public HoodieFileFormat getLogFileFormat() {
return metaClient.getTableConfig().getLogFileFormat();
}
- public HoodieLogBlockType getLogDataBlockFormat() {
Review comment:
this is a very good cleanup. thanks!
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -1503,6 +1504,11 @@ public String parquetOutputTimestampType() {
return getString(HoodieStorageConfig.PARQUET_OUTPUT_TIMESTAMP_TYPE);
}
+ public HoodieLogBlock.HoodieLogBlockType getLogDataBlockFormat() {
Review comment:
Can we avoid `null` as the sentinel here and use an `Option`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,151 +18,173 @@
package org.apache.hudi.common.table.log.block;
-import org.apache.hudi.common.model.HoodieRecord;
-import org.apache.hudi.common.util.Option;
-import org.apache.hudi.exception.HoodieException;
-import org.apache.hudi.exception.HoodieIOException;
-
import org.apache.avro.Schema;
import org.apache.avro.generic.IndexedRecord;
import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private Option<List<IndexedRecord>> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
- public HoodieDataBlock(@Nonnull List<IndexedRecord> records, @Nonnull Map<HeaderMetadataType, String> header,
- @Nonnull Map<HeaderMetadataType, String> footer, String keyField) {
- this(header, footer, Option.empty(), Option.empty(), null, false);
- this.records = records;
- this.schema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- this.keyField = keyField;
- }
+ private final boolean enablePointLookups;
- protected HoodieDataBlock(Option<byte[]> content, @Nonnull FSDataInputStream inputStream, boolean readBlockLazily,
- Option<HoodieLogBlockContentLocation> blockContentLocation, Schema readerSchema,
- @Nonnull Map<HeaderMetadataType, String> headers, @Nonnull Map<HeaderMetadataType,
- String> footer, String keyField) {
- this(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
- this.schema = readerSchema;
- this.keyField = keyField;
- }
+ protected final Schema readerSchema;
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header) {
- return getBlock(logDataBlockFormat, recordList, header, HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ public HoodieDataBlock(List<IndexedRecord> records,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef) {
+ super(header, footer, Option.empty(), Option.empty(), null, false);
+ this.records = Option.of(records);
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = getWriterSchema(super.getLogBlockHeader());
+ this.enablePointLookups = false;
}
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @param keyField - FieldId to get the key from the records
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header, String keyField) {
- switch (logDataBlockFormat) {
- case AVRO_DATA_BLOCK:
- return new HoodieAvroDataBlock(recordList, header, keyField);
- case HFILE_DATA_BLOCK:
- return new HoodieHFileDataBlock(recordList, header, keyField);
- default:
- throw new HoodieException("Data block format " + logDataBlockFormat + " not implemented");
- }
+ protected HoodieDataBlock(Option<byte[]> content,
+ FSDataInputStream inputStream,
+ boolean readBlockLazily,
+ Option<HoodieLogBlockContentLocation> blockContentLocation,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> headers,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef,
+ boolean enablePointLookups) {
+ super(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
+ this.records = Option.empty();
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = readerSchema.orElseGet(() -> getWriterSchema(super.getLogBlockHeader()));
+ this.enablePointLookups = enablePointLookups;
}
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records.isPresent(), "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records.get());
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
- if (records == null) {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
+ if (!records.isPresent()) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = Option.of(readRecordsFromBlockPayload());
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
- return records;
+ return records.get();
+ }
+
+ public Schema getSchema() {
+ return readerSchema;
}
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
+ }
+ }
+
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ throw new UnsupportedOperationException(
+ String.format("Point lookups are not supported by this Data block type (%s)", getBlockType())
+ );
}
- protected abstract byte[] serializeRecords() throws IOException;
+ protected abstract byte[] serializeRecords(List<IndexedRecord> records) throws IOException;
- protected abstract void deserializeRecords() throws IOException;
+ protected abstract List<IndexedRecord> deserializeRecords(byte[] content) throws IOException;
+
+ public abstract HoodieLogBlockType getBlockType();
+
+ protected String getRecordKey(IndexedRecord record) {
+ return Option.ofNullable(record.getSchema().getField(keyFieldRef))
+ .map(keyField -> record.get(keyField.pos()))
Review comment:
there is similar code for the metadata table/key field de-dupe work. Could we carve this into some util method
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieParquetStreamWriter.java
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.parquet.io.OutputStreamBackedOutputFile;
+import org.apache.parquet.hadoop.ParquetFileWriter;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.api.WriteSupport;
+import org.apache.parquet.io.OutputFile;
+
+import java.io.IOException;
+
+// TODO unify w/ HoodieParquetWriter
Review comment:
JIRA for this?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,151 +18,173 @@
package org.apache.hudi.common.table.log.block;
-import org.apache.hudi.common.model.HoodieRecord;
-import org.apache.hudi.common.util.Option;
-import org.apache.hudi.exception.HoodieException;
-import org.apache.hudi.exception.HoodieIOException;
-
import org.apache.avro.Schema;
import org.apache.avro.generic.IndexedRecord;
import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private Option<List<IndexedRecord>> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
- public HoodieDataBlock(@Nonnull List<IndexedRecord> records, @Nonnull Map<HeaderMetadataType, String> header,
- @Nonnull Map<HeaderMetadataType, String> footer, String keyField) {
- this(header, footer, Option.empty(), Option.empty(), null, false);
- this.records = records;
- this.schema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- this.keyField = keyField;
- }
+ private final boolean enablePointLookups;
- protected HoodieDataBlock(Option<byte[]> content, @Nonnull FSDataInputStream inputStream, boolean readBlockLazily,
- Option<HoodieLogBlockContentLocation> blockContentLocation, Schema readerSchema,
- @Nonnull Map<HeaderMetadataType, String> headers, @Nonnull Map<HeaderMetadataType,
- String> footer, String keyField) {
- this(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
- this.schema = readerSchema;
- this.keyField = keyField;
- }
+ protected final Schema readerSchema;
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header) {
- return getBlock(logDataBlockFormat, recordList, header, HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ public HoodieDataBlock(List<IndexedRecord> records,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef) {
+ super(header, footer, Option.empty(), Option.empty(), null, false);
+ this.records = Option.of(records);
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = getWriterSchema(super.getLogBlockHeader());
+ this.enablePointLookups = false;
}
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @param keyField - FieldId to get the key from the records
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header, String keyField) {
- switch (logDataBlockFormat) {
- case AVRO_DATA_BLOCK:
- return new HoodieAvroDataBlock(recordList, header, keyField);
- case HFILE_DATA_BLOCK:
- return new HoodieHFileDataBlock(recordList, header, keyField);
- default:
- throw new HoodieException("Data block format " + logDataBlockFormat + " not implemented");
- }
+ protected HoodieDataBlock(Option<byte[]> content,
+ FSDataInputStream inputStream,
+ boolean readBlockLazily,
+ Option<HoodieLogBlockContentLocation> blockContentLocation,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> headers,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef,
+ boolean enablePointLookups) {
+ super(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
+ this.records = Option.empty();
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = readerSchema.orElseGet(() -> getWriterSchema(super.getLogBlockHeader()));
+ this.enablePointLookups = enablePointLookups;
}
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records.isPresent(), "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records.get());
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
- if (records == null) {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
+ if (!records.isPresent()) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = Option.of(readRecordsFromBlockPayload());
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
- return records;
+ return records.get();
+ }
+
+ public Schema getSchema() {
+ return readerSchema;
}
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
+ }
+ }
+
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ throw new UnsupportedOperationException(
+ String.format("Point lookups are not supported by this Data block type (%s)", getBlockType())
+ );
}
- protected abstract byte[] serializeRecords() throws IOException;
+ protected abstract byte[] serializeRecords(List<IndexedRecord> records) throws IOException;
- protected abstract void deserializeRecords() throws IOException;
+ protected abstract List<IndexedRecord> deserializeRecords(byte[] content) throws IOException;
+
+ public abstract HoodieLogBlockType getBlockType();
+
+ protected String getRecordKey(IndexedRecord record) {
+ return Option.ofNullable(record.getSchema().getField(keyFieldRef))
+ .map(keyField -> record.get(keyField.pos()))
+ .map(Object::toString)
+ .orElse(null);
Review comment:
avoid returning null?
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieParquetStreamWriter.java
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.parquet.io.OutputStreamBackedOutputFile;
+import org.apache.parquet.hadoop.ParquetFileWriter;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.api.WriteSupport;
+import org.apache.parquet.io.OutputFile;
+
+import java.io.IOException;
+
+// TODO unify w/ HoodieParquetWriter
+public class HoodieParquetStreamWriter<R extends IndexedRecord> implements AutoCloseable {
+
+ private final ParquetWriter<R> writer;
+ private final HoodieAvroWriteSupport writeSupport;
+
+ public HoodieParquetStreamWriter(
+ FSDataOutputStream outputStream,
+ HoodieAvroParquetConfig parquetConfig) throws IOException {
+ this.writeSupport = parquetConfig.getWriteSupport();
+ this.writer = new Builder<R>(new OutputStreamBackedOutputFile(outputStream), writeSupport)
Review comment:
do we set all the configs that are currently set on the other path. double check once?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,151 +18,173 @@
package org.apache.hudi.common.table.log.block;
-import org.apache.hudi.common.model.HoodieRecord;
-import org.apache.hudi.common.util.Option;
-import org.apache.hudi.exception.HoodieException;
-import org.apache.hudi.exception.HoodieIOException;
-
import org.apache.avro.Schema;
import org.apache.avro.generic.IndexedRecord;
import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private Option<List<IndexedRecord>> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
Review comment:
rename: keyFieldName?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -481,4 +447,63 @@ public long moveToPrev() throws IOException {
public void remove() {
throw new UnsupportedOperationException("Remove not supported for HoodieLogFileReader");
}
+
+ private static Path makeQualified(FileSystem fs, Path path) {
Review comment:
move to FSUtils
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/model/HoodieLogFile.java
##########
@@ -60,7 +59,7 @@ public HoodieLogFile(FileStatus fileStatus) {
public HoodieLogFile(Path logPath) {
this.fileStatus = null;
this.pathStr = logPath.toString();
- this.fileLen = 0;
+ this.fileLen = -1;
Review comment:
did you hit any issues that required this to be -1. I am okay with the change. But there might be side-effects to consider?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,151 +18,173 @@
package org.apache.hudi.common.table.log.block;
-import org.apache.hudi.common.model.HoodieRecord;
-import org.apache.hudi.common.util.Option;
-import org.apache.hudi.exception.HoodieException;
-import org.apache.hudi.exception.HoodieIOException;
-
import org.apache.avro.Schema;
import org.apache.avro.generic.IndexedRecord;
import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private Option<List<IndexedRecord>> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
- public HoodieDataBlock(@Nonnull List<IndexedRecord> records, @Nonnull Map<HeaderMetadataType, String> header,
- @Nonnull Map<HeaderMetadataType, String> footer, String keyField) {
- this(header, footer, Option.empty(), Option.empty(), null, false);
- this.records = records;
- this.schema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- this.keyField = keyField;
- }
+ private final boolean enablePointLookups;
- protected HoodieDataBlock(Option<byte[]> content, @Nonnull FSDataInputStream inputStream, boolean readBlockLazily,
- Option<HoodieLogBlockContentLocation> blockContentLocation, Schema readerSchema,
- @Nonnull Map<HeaderMetadataType, String> headers, @Nonnull Map<HeaderMetadataType,
- String> footer, String keyField) {
- this(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
- this.schema = readerSchema;
- this.keyField = keyField;
- }
+ protected final Schema readerSchema;
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header) {
- return getBlock(logDataBlockFormat, recordList, header, HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ public HoodieDataBlock(List<IndexedRecord> records,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef) {
+ super(header, footer, Option.empty(), Option.empty(), null, false);
+ this.records = Option.of(records);
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = getWriterSchema(super.getLogBlockHeader());
+ this.enablePointLookups = false;
}
/**
- * Util method to get a data block for the requested type.
- *
- * @param logDataBlockFormat - Data block type
- * @param recordList - List of records that goes in the data block
- * @param header - data block header
- * @param keyField - FieldId to get the key from the records
- * @return Data block of the requested type.
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
*/
- public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, List<IndexedRecord> recordList,
- Map<HeaderMetadataType, String> header, String keyField) {
- switch (logDataBlockFormat) {
- case AVRO_DATA_BLOCK:
- return new HoodieAvroDataBlock(recordList, header, keyField);
- case HFILE_DATA_BLOCK:
- return new HoodieHFileDataBlock(recordList, header, keyField);
- default:
- throw new HoodieException("Data block format " + logDataBlockFormat + " not implemented");
- }
+ protected HoodieDataBlock(Option<byte[]> content,
+ FSDataInputStream inputStream,
+ boolean readBlockLazily,
+ Option<HoodieLogBlockContentLocation> blockContentLocation,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> headers,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef,
+ boolean enablePointLookups) {
+ super(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
+ this.records = Option.empty();
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = readerSchema.orElseGet(() -> getWriterSchema(super.getLogBlockHeader()));
+ this.enablePointLookups = enablePointLookups;
}
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records.isPresent(), "Block is in invalid state");
Review comment:
add more reasoning to the error message
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -224,6 +217,33 @@ public static HoodieAvroDataBlock getBlock(byte[] content, Schema readerSchema)
return new HoodieAvroDataBlock(records, readerSchema);
}
+ private static byte[] compress(String text) {
Review comment:
and add an unit test around it
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -224,6 +217,33 @@ public static HoodieAvroDataBlock getBlock(byte[] content, Schema readerSchema)
return new HoodieAvroDataBlock(records, readerSchema);
}
+ private static byte[] compress(String text) {
Review comment:
can we move this to IOUtils or some common utils class?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -481,4 +447,63 @@ public long moveToPrev() throws IOException {
public void remove() {
throw new UnsupportedOperationException("Remove not supported for HoodieLogFileReader");
}
+
+ private static Path makeQualified(FileSystem fs, Path path) {
+ return path.makeQualified(fs.getUri(), fs.getWorkingDirectory());
+ }
+
+ /**
+ * Fetch the right {@link FSDataInputStream} to be used by wrapping with required input streams.
+ * @param fs instance of {@link FileSystem} in use.
+ * @param bufferSize buffer size to be used.
+ * @return the right {@link FSDataInputStream} as required.
+ */
+ private static FSDataInputStream getFSDataInputStream(FileSystem fs,
Review comment:
whenever you move code, please leave a comment on the PR on what has changed. Can you let me know if any code has changed here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r773291825
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/ByteBufferBackedInputFile.java
##########
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hudi.common.util.io.ByteBufferBackedInputStream;
+import org.apache.parquet.io.DelegatingSeekableInputStream;
+import org.apache.parquet.io.InputFile;
+import org.apache.parquet.io.SeekableInputStream;
+
+/**
+ * Implementation of {@link InputFile} backed by {@code byte[]} buffer
+ */
+public class ByteBufferBackedInputFile implements InputFile {
+ private final byte[] buffer;
+ private final int offset;
+ private final int length;
+
+ public ByteBufferBackedInputFile(byte[] buffer, int offset, int length) {
+ this.buffer = buffer;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public ByteBufferBackedInputFile(byte[] buffer) {
+ this(buffer, 0, buffer.length);
+ }
+
+ @Override
+ public long getLength() {
+ return length;
+ }
+
+ @Override
+ public SeekableInputStream newStream() {
Review comment:
These are covered by existing tests.
There was previously coverage report added to PRs, wasn't there? What happened to it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012727208
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012785174
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997091597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995334814
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996995275
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999335468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999873768
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
* 573af13b233334d2678ab40ca51c7424757aae60 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999887877
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999917098
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995350274
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995350274
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996226567
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996260907
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005356347
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006099093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r784094553
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -125,6 +126,11 @@
.withAlternatives("hoodie.table.rt.file.format")
.withDocumentation("Log format used for the delta logs.");
+ public static final ConfigProperty<String> LOG_BLOCK_TYPE = ConfigProperty
Review comment:
yes. it will be a write config. We can to facilitate newer file groups written out as avro from say Kafka Connect, while a delete job adds bulk deletes/updates into log blocks in parquet.
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##########
@@ -719,7 +720,13 @@ public HoodieFileFormat getLogFileFormat() {
return metaClient.getTableConfig().getLogFileFormat();
}
- public HoodieLogBlockType getLogDataBlockFormat() {
+ public HoodieLogBlockType getLogDataBlockType() {
+ HoodieLogBlock.HoodieLogBlockType logBlockType = metaClient.getTableConfig().getLogBlockFormat();
Review comment:
We need to move the log block format away from the table config
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java
##########
@@ -497,4 +489,32 @@ private void flushToDiskIfRequired(HoodieRecord record) {
numberOfRecords = 0;
}
}
+
+ protected void appendDataAndDeleteBlocks(Map<HeaderMetadataType, String> header) {
Review comment:
@alexeykudinkin can you call out what has changed in this block of code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006146863
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012786245
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996238336
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995334814
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995336374
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995366619
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995336374
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005414443
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006113171
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012701103
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012774985
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027483972
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #4333:
URL: https://github.com/apache/hudi/pull/4333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995438872
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998321139
@vinothchandar i've added a test asserting that for Parquet we only read projected columns
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r774250575
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
nothing specific. just incase for parquet reader/writer, we intend to pass anything from higher layers, we need to be mindful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996260907
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997091597
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995438872
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996205337
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995489753
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998347129
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r773301245
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
We don't pass Hadoop Conf into the Data blocks, since we pass already open File Stream, and it was this way for HFile already so i've simply carried it forward w/o much afterthought.
What kind of config do you have in mind that we might miss?
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/OutputStreamBackedOutputFile.java
##########
@@ -0,0 +1,94 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.parquet.io.OutputFile;
+import org.apache.parquet.io.PositionOutputStream;
+
+import javax.annotation.Nonnull;
+import java.io.IOException;
+
+/**
+ * Implementation of the {@link OutputFile} backed by {@link java.io.OutputStream}
+ */
+public class OutputStreamBackedOutputFile implements OutputFile {
Review comment:
These are inspired by the need to implement `OutputFile` :-)
Flink is one of those impls
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997144563
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r774314975
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -88,76 +92,24 @@ public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSc
}
public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSchema, int bufferSize,
- boolean readBlockLazily, boolean reverseReader, boolean enableInlineReading,
+ boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups,
String keyField) throws IOException {
- FSDataInputStream fsDataInputStream = fs.open(logFile.getPath(), bufferSize);
- this.logFile = logFile;
- this.inputStream = getFSDataInputStream(fsDataInputStream, fs, bufferSize);
+ // NOTE: We repackage {@code HoodieLogFile} here to make sure that the provided path
+ // is prefixed with an appropriate scheme given that we're not propagating the FS
+ // further
+ this.logFile = new HoodieLogFile(fs.getFileStatus(logFile.getPath()));
Review comment:
let'sgetrid of the extra fs. getFileStatus call. Its an extra Rpc to S3/HDFS
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -481,4 +442,59 @@ public long moveToPrev() throws IOException {
public void remove() {
throw new UnsupportedOperationException("Remove not supported for HoodieLogFileReader");
}
+
+ /**
+ * Fetch the right {@link FSDataInputStream} to be used by wrapping with required input streams.
+ * @param fs instance of {@link FileSystem} in use.
+ * @param bufferSize buffer size to be used.
+ * @return the right {@link FSDataInputStream} as required.
+ */
+ private static FSDataInputStream getFSDataInputStream(FileSystem fs,
Review comment:
has any code Changed here?
Please refrain from relocating code in the same PR that also changes/rewrites actual logic. If you must, then annotate the PR with comments letting the reviewer know what's going on .
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -288,28 +288,14 @@ public synchronized void close() {
}
}
- static class SeekableByteArrayInputStream extends ByteArrayInputStream implements Seekable, PositionedReadable {
+ static class SeekableByteArrayInputStream extends ByteBufferBackedInputStream implements Seekable, PositionedReadable {
Review comment:
something that changes the reading for HFile in general, should go on its own PR?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -72,7 +76,7 @@
private long reverseLogFilePosition;
private long lastReverseLogFilePosition;
private boolean reverseReader;
- private boolean enableInlineReading;
+ private boolean enableRecordLookups;
Review comment:
Wondering if there are actually two separate concepts here.
inline reading - Where are use a parquet reader or hfile reader to read block content, iinstead of reading it out as a byte[]
record lookups - Whether we scan and filter the keys of interest or push the lookups to the reader/fileformat.
I feel the first one should be what the log reader should care about. We should push the record lookups into the file reader?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -88,76 +92,24 @@ public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSc
}
public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSchema, int bufferSize,
- boolean readBlockLazily, boolean reverseReader, boolean enableInlineReading,
+ boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups,
String keyField) throws IOException {
- FSDataInputStream fsDataInputStream = fs.open(logFile.getPath(), bufferSize);
- this.logFile = logFile;
- this.inputStream = getFSDataInputStream(fsDataInputStream, fs, bufferSize);
+ // NOTE: We repackage {@code HoodieLogFile} here to make sure that the provided path
+ // is prefixed with an appropriate scheme given that we're not propagating the FS
+ // further
+ this.logFile = new HoodieLogFile(fs.getFileStatus(logFile.getPath()));
Review comment:
In general, there should be no direct list or status fetches here. We removed all of them with O.9.0
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -208,71 +155,85 @@ private HoodieLogBlock readBlock() throws IOException {
HoodieLogFormat.LogFormatVersion nextBlockVersion = readVersion();
// 3. Read the block type for a log block
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- type = inputStream.readInt();
-
- ValidationUtils.checkArgument(type < HoodieLogBlockType.values().length, "Invalid block byte type found " + type);
- blockType = HoodieLogBlockType.values()[type];
- }
+ HoodieLogBlockType blockType = tryReadBlockType(nextBlockVersion);
// 4. Read the header for a log block, if present
- if (nextBlockVersion.hasHeader()) {
- header = HoodieLogBlock.getLogMetadata(inputStream);
- }
- int contentLength = blocksize;
+ Map<HeaderMetadataType, String> header =
+ nextBlockVersion.hasHeader() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
+
// 5. Read the content length for the content
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- contentLength = (int) inputStream.readLong();
- }
+ // Fallback to full-block size if no content-length
+ // TODO replace w/ hasContentLength
+ int contentLength =
+ nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION ? (int) inputStream.readLong() : blockSize;
// 6. Read the content or skip content based on IO vs Memory trade-off by client
- // TODO - have a max block size and reuse this buffer in the ByteBuffer
- // (hard to guess max block size for now)
long contentPosition = inputStream.getPos();
- byte[] content = HoodieLogBlock.readOrSkipContent(inputStream, contentLength, readBlockLazily);
+ boolean shouldReadLazily = readBlockLazily && nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION;
+ Option<byte[]> content = HoodieLogBlock.tryReadContent(inputStream, contentLength, shouldReadLazily);
// 7. Read footer if any
- Map<HeaderMetadataType, String> footer = null;
- if (nextBlockVersion.hasFooter()) {
- footer = HoodieLogBlock.getLogMetadata(inputStream);
- }
+ Map<HeaderMetadataType, String> footer =
+ nextBlockVersion.hasFooter() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
// 8. Read log block length, if present. This acts as a reverse pointer when traversing a
// log file in reverse
- @SuppressWarnings("unused")
- long logBlockLength = 0;
if (nextBlockVersion.hasLogBlockLength()) {
- logBlockLength = inputStream.readLong();
+ inputStream.readLong();
}
// 9. Read the log block end position in the log file
long blockEndPos = inputStream.getPos();
switch (Objects.requireNonNull(blockType)) {
- // based on type read the block
case AVRO_DATA_BLOCK:
if (nextBlockVersion.getVersion() == HoodieLogFormatVersion.DEFAULT_VERSION) {
- return HoodieAvroDataBlock.getBlock(content, readerSchema);
+ return HoodieAvroDataBlock.getBlock(content.get(), readerSchema);
} else {
- return new HoodieAvroDataBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
- contentPosition, contentLength, blockEndPos, readerSchema, header, footer, keyField);
+ return new HoodieAvroDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema), header, footer, keyField);
}
+
case HFILE_DATA_BLOCK:
- return new HoodieHFileDataBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
- contentPosition, contentLength, blockEndPos, readerSchema,
- header, footer, enableInlineReading, keyField);
+ checkState(nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION,
+ String.format("HFile block could not be of version (%d)", HoodieLogFormatVersion.DEFAULT_VERSION));
+
+ return new HoodieHFileDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema),
+ header, footer, keyField, enableRecordLookups);
+
+ case PARQUET_DATA_BLOCK:
+ checkState(nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION,
+ String.format("Parquet block could not be of version (%d)", HoodieLogFormatVersion.DEFAULT_VERSION));
+
+ return new HoodieParquetDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema), header, footer, keyField);
+
case DELETE_BLOCK:
- return HoodieDeleteBlock.getBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
+ return HoodieDeleteBlock.getBlock(logFile, inputStream, content, readBlockLazily,
contentPosition, contentLength, blockEndPos, header, footer);
+
case COMMAND_BLOCK:
- return HoodieCommandBlock.getBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
+ return HoodieCommandBlock.getBlock(logFile, inputStream, content, readBlockLazily,
contentPosition, contentLength, blockEndPos, header, footer);
+
default:
throw new HoodieNotSupportedException("Unsupported Block " + blockType);
}
}
+ @Nullable
+ private HoodieLogBlockType tryReadBlockType(HoodieLogFormat.LogFormatVersion blockVersion) throws IOException {
+ if (blockVersion.getVersion() == HoodieLogFormatVersion.DEFAULT_VERSION) {
+ return null;
Review comment:
The nulls as sentinel here makes one nervous. Can we use Option?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -208,71 +155,85 @@ private HoodieLogBlock readBlock() throws IOException {
HoodieLogFormat.LogFormatVersion nextBlockVersion = readVersion();
// 3. Read the block type for a log block
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- type = inputStream.readInt();
-
- ValidationUtils.checkArgument(type < HoodieLogBlockType.values().length, "Invalid block byte type found " + type);
- blockType = HoodieLogBlockType.values()[type];
- }
+ HoodieLogBlockType blockType = tryReadBlockType(nextBlockVersion);
// 4. Read the header for a log block, if present
- if (nextBlockVersion.hasHeader()) {
- header = HoodieLogBlock.getLogMetadata(inputStream);
- }
- int contentLength = blocksize;
+ Map<HeaderMetadataType, String> header =
+ nextBlockVersion.hasHeader() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
+
// 5. Read the content length for the content
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- contentLength = (int) inputStream.readLong();
- }
+ // Fallback to full-block size if no content-length
+ // TODO replace w/ hasContentLength
+ int contentLength =
+ nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION ? (int) inputStream.readLong() : blockSize;
// 6. Read the content or skip content based on IO vs Memory trade-off by client
- // TODO - have a max block size and reuse this buffer in the ByteBuffer
- // (hard to guess max block size for now)
long contentPosition = inputStream.getPos();
- byte[] content = HoodieLogBlock.readOrSkipContent(inputStream, contentLength, readBlockLazily);
+ boolean shouldReadLazily = readBlockLazily && nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION;
+ Option<byte[]> content = HoodieLogBlock.tryReadContent(inputStream, contentLength, shouldReadLazily);
// 7. Read footer if any
- Map<HeaderMetadataType, String> footer = null;
- if (nextBlockVersion.hasFooter()) {
- footer = HoodieLogBlock.getLogMetadata(inputStream);
- }
+ Map<HeaderMetadataType, String> footer =
Review comment:
Can we work with empty map instead of null?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlock.java
##########
@@ -109,7 +116,25 @@ public long getLogBlockLength() {
* Type of the log block WARNING: This enum is serialized as the ordinal. Only add new enums at the end.
*/
public enum HoodieLogBlockType {
- COMMAND_BLOCK, DELETE_BLOCK, CORRUPT_BLOCK, AVRO_DATA_BLOCK, HFILE_DATA_BLOCK
+ COMMAND_BLOCK(":command"),
+ DELETE_BLOCK(":delete"),
+ CORRUPT_BLOCK(":corrupted"),
Review comment:
rename: "corrupt" its a valid adjective. Match the enum name ?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -58,24 +57,37 @@
import java.util.TreeMap;
import java.util.stream.Collectors;
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
+
/**
* HoodieHFileDataBlock contains a list of records stored inside an HFile format. It is used with the HFile
* base file format.
*/
public class HoodieHFileDataBlock extends HoodieDataBlock {
private static final Logger LOG = LogManager.getLogger(HoodieHFileDataBlock.class);
- private static Compression.Algorithm compressionAlgorithm = Compression.Algorithm.GZ;
- private static int blockSize = 1 * 1024 * 1024;
- private boolean enableInlineReading = false;
-
- public HoodieHFileDataBlock(HoodieLogFile logFile, FSDataInputStream inputStream, Option<byte[]> content,
- boolean readBlockLazily, long position, long blockSize, long blockEndpos,
- Schema readerSchema, Map<HeaderMetadataType, String> header,
- Map<HeaderMetadataType, String> footer, boolean enableInlineReading, String keyField) {
- super(content, inputStream, readBlockLazily,
- Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
- readerSchema, header, footer, keyField);
- this.enableInlineReading = enableInlineReading;
+
+ private static final int DEFAULT_BLOCK_SIZE = 1024 * 1024;
+ private static final Compression.Algorithm DEFAULT_COMPRESSION_ALGO = Compression.Algorithm.GZ;
+
+ public HoodieHFileDataBlock(HoodieLogFile logFile,
Review comment:
let's revert back to the wrapped style.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -88,76 +92,24 @@ public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSc
}
public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSchema, int bufferSize,
- boolean readBlockLazily, boolean reverseReader, boolean enableInlineReading,
+ boolean readBlockLazily, boolean reverseReader, boolean enableRecordLookups,
String keyField) throws IOException {
- FSDataInputStream fsDataInputStream = fs.open(logFile.getPath(), bufferSize);
- this.logFile = logFile;
- this.inputStream = getFSDataInputStream(fsDataInputStream, fs, bufferSize);
+ // NOTE: We repackage {@code HoodieLogFile} here to make sure that the provided path
+ // is prefixed with an appropriate scheme given that we're not propagating the FS
+ // further
+ this.logFile = new HoodieLogFile(fs.getFileStatus(logFile.getPath()));
+ this.inputStream = getFSDataInputStream(fs, this.logFile, bufferSize);
this.readerSchema = readerSchema;
this.readBlockLazily = readBlockLazily;
this.reverseReader = reverseReader;
- this.enableInlineReading = enableInlineReading;
+ this.enableRecordLookups = enableRecordLookups;
this.keyField = keyField;
if (this.reverseReader) {
- this.reverseLogFilePosition = this.lastReverseLogFilePosition = logFile.getFileSize();
+ this.reverseLogFilePosition = this.lastReverseLogFilePosition = this.logFile.getFileSize();
}
addShutDownHook();
}
- public HoodieLogFileReader(FileSystem fs, HoodieLogFile logFile, Schema readerSchema) throws IOException {
- this(fs, logFile, readerSchema, DEFAULT_BUFFER_SIZE, false, false);
- }
-
- /**
- * Fetch the right {@link FSDataInputStream} to be used by wrapping with required input streams.
- * @param fsDataInputStream original instance of {@link FSDataInputStream}.
- * @param fs instance of {@link FileSystem} in use.
- * @param bufferSize buffer size to be used.
- * @return the right {@link FSDataInputStream} as required.
- */
- private FSDataInputStream getFSDataInputStream(FSDataInputStream fsDataInputStream, FileSystem fs, int bufferSize) {
Review comment:
Where is all this moved to? (note to self)
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -148,65 +165,40 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected void createRecordsFromContentBytes() throws IOException {
- if (enableInlineReading) {
- getRecords(Collections.emptyList());
- } else {
- super.createRecordsFromContentBytes();
- }
- }
-
- @Override
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- readWithInlineFS(keys);
- return records;
- }
-
- private void readWithInlineFS(List<String> keys) throws IOException {
- boolean enableFullScan = keys.isEmpty();
- // Get schema from the header
- Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- // If readerSchema was not present, use writerSchema
- if (schema == null) {
- schema = writerSchema;
- }
- Configuration conf = new Configuration();
- CacheConfig cacheConf = new CacheConfig(conf);
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
Configuration inlineConf = new Configuration();
inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
Path inlinePath = InLineFSUtils.getInlineFilePath(
- getBlockContentLocation().get().getLogFile().getPath(),
- getBlockContentLocation().get().getLogFile().getPath().getFileSystem(conf).getScheme(),
- getBlockContentLocation().get().getContentPositionInLogFile(),
- getBlockContentLocation().get().getBlockSize());
- if (!enableFullScan) {
- // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
- Collections.sort(keys);
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
+ Collections.sort(keys);
+
+ try (HoodieHFileReader<IndexedRecord> reader =
+ new HoodieHFileReader<>(inlineConf, inlinePath, new CacheConfig(inlineConf), inlinePath.getFileSystem(inlineConf))) {
+ // Get writer's schema from the header
+ List<Pair<String, IndexedRecord>> logRecords = reader.readRecords(keys, readerSchema);
+ return logRecords.stream().map(Pair::getSecond).collect(Collectors.toList());
}
- HoodieHFileReader reader = new HoodieHFileReader(inlineConf, inlinePath, cacheConf, inlinePath.getFileSystem(inlineConf));
- List<org.apache.hadoop.hbase.util.Pair<String, IndexedRecord>> logRecords = enableFullScan ? reader.readAllRecords(writerSchema, schema) :
Review comment:
We should just be using our own Pair.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlock.java
##########
@@ -109,7 +116,25 @@ public long getLogBlockLength() {
* Type of the log block WARNING: This enum is serialized as the ordinal. Only add new enums at the end.
*/
public enum HoodieLogBlockType {
- COMMAND_BLOCK, DELETE_BLOCK, CORRUPT_BLOCK, AVRO_DATA_BLOCK, HFILE_DATA_BLOCK
+ COMMAND_BLOCK(":command"),
+ DELETE_BLOCK(":delete"),
+ CORRUPT_BLOCK(":corrupted"),
Review comment:
rename: "corrupt" its a valid adjective. Match the enum name ?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -148,65 +165,40 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected void createRecordsFromContentBytes() throws IOException {
- if (enableInlineReading) {
- getRecords(Collections.emptyList());
- } else {
- super.createRecordsFromContentBytes();
- }
- }
-
- @Override
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- readWithInlineFS(keys);
- return records;
- }
-
- private void readWithInlineFS(List<String> keys) throws IOException {
- boolean enableFullScan = keys.isEmpty();
- // Get schema from the header
- Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- // If readerSchema was not present, use writerSchema
- if (schema == null) {
- schema = writerSchema;
- }
- Configuration conf = new Configuration();
- CacheConfig cacheConf = new CacheConfig(conf);
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
Configuration inlineConf = new Configuration();
inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
Path inlinePath = InLineFSUtils.getInlineFilePath(
- getBlockContentLocation().get().getLogFile().getPath(),
- getBlockContentLocation().get().getLogFile().getPath().getFileSystem(conf).getScheme(),
- getBlockContentLocation().get().getContentPositionInLogFile(),
- getBlockContentLocation().get().getBlockSize());
- if (!enableFullScan) {
- // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
- Collections.sort(keys);
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
+ Collections.sort(keys);
+
+ try (HoodieHFileReader<IndexedRecord> reader =
+ new HoodieHFileReader<>(inlineConf, inlinePath, new CacheConfig(inlineConf), inlinePath.getFileSystem(inlineConf))) {
+ // Get writer's schema from the header
+ List<Pair<String, IndexedRecord>> logRecords = reader.readRecords(keys, readerSchema);
+ return logRecords.stream().map(Pair::getSecond).collect(Collectors.toList());
}
- HoodieHFileReader reader = new HoodieHFileReader(inlineConf, inlinePath, cacheConf, inlinePath.getFileSystem(inlineConf));
- List<org.apache.hadoop.hbase.util.Pair<String, IndexedRecord>> logRecords = enableFullScan ? reader.readAllRecords(writerSchema, schema) :
- reader.readRecords(keys, schema);
- reader.close();
- this.records = logRecords.stream().map(t -> t.getSecond()).collect(Collectors.toList());
}
@Override
- protected void deserializeRecords() throws IOException {
+ protected List<IndexedRecord> deserializeRecords(byte[] content) throws IOException {
+ checkState(readerSchema != null, "Reader's schema has to be non-null");
Review comment:
Is this because of change in the PR? the original code seems to expect null reader schema at times?
##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HadoopMapRedUtils.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.testutils;
+
+import org.apache.hadoop.mapred.Counters;
+import org.apache.hadoop.mapred.InputSplit;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hudi.common.util.Option;
+
+import java.util.concurrent.ConcurrentHashMap;
+
+public class HadoopMapRedUtils {
Review comment:
Could we rewrite the tests in a different way, than relying on the mapred classes. generally, they are a world of pain as we test across hadoop versions
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
If someone manually sets `spark.sparkContext.hadoopConfiguration.set(x, y)` say configuring cloud access creds, this is going to be an issue, since `new Configuration()` won't pick that up
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/util/io/ByteBufferBackedInputStream.java
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.util.io;
+
+import javax.annotation.Nonnull;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+
+/**
+ * Instance of {@link InputStream} backed by {@link ByteBuffer}, implementing following
+ * functionality (on top of what's required by {@link InputStream})
+ *
+ * <ol>
+ * <li>Seeking: enables random access by allowing to seek to an arbitrary position w/in the stream</li>
+ * <li>(Thread-safe) Copying: enables to copy from the underlying buffer not modifying the state of the stream</li>
+ * </ol>
+ *
+ * NOTE: Generally methods of this class are NOT thread-safe, unless specified otherwise
+ */
+public class ByteBufferBackedInputStream extends InputStream {
Review comment:
can we UT this class?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlock.java
##########
@@ -109,7 +116,25 @@ public long getLogBlockLength() {
* Type of the log block WARNING: This enum is serialized as the ordinal. Only add new enums at the end.
*/
public enum HoodieLogBlockType {
- COMMAND_BLOCK, DELETE_BLOCK, CORRUPT_BLOCK, AVRO_DATA_BLOCK, HFILE_DATA_BLOCK
+ COMMAND_BLOCK(":command"),
Review comment:
why the :, Why do we need the id?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
+1 , given we are cleaning up a bunch here, lets also fix this. We need to wire in the config from the TableMetaClient into this layer.
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieParquetStreamWriter.java
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.parquet.io.OutputStreamBackedOutputFile;
+import org.apache.parquet.hadoop.ParquetFileWriter;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.api.WriteSupport;
+import org.apache.parquet.io.OutputFile;
+
+import java.io.IOException;
+
+// TODO unify w/ HoodieParquetWriter
+public class HoodieParquetStreamWriter<R extends IndexedRecord> implements AutoCloseable {
Review comment:
why do we need this separate class? Why not enhance `HoodieParquetWriter`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -148,65 +165,40 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected void createRecordsFromContentBytes() throws IOException {
- if (enableInlineReading) {
- getRecords(Collections.emptyList());
- } else {
- super.createRecordsFromContentBytes();
- }
- }
-
- @Override
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- readWithInlineFS(keys);
- return records;
- }
-
- private void readWithInlineFS(List<String> keys) throws IOException {
- boolean enableFullScan = keys.isEmpty();
- // Get schema from the header
- Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- // If readerSchema was not present, use writerSchema
- if (schema == null) {
- schema = writerSchema;
- }
- Configuration conf = new Configuration();
- CacheConfig cacheConf = new CacheConfig(conf);
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
Configuration inlineConf = new Configuration();
inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
Path inlinePath = InLineFSUtils.getInlineFilePath(
- getBlockContentLocation().get().getLogFile().getPath(),
- getBlockContentLocation().get().getLogFile().getPath().getFileSystem(conf).getScheme(),
- getBlockContentLocation().get().getContentPositionInLogFile(),
- getBlockContentLocation().get().getBlockSize());
- if (!enableFullScan) {
- // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
- Collections.sort(keys);
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ // HFile read will be efficient if keys are sorted, since on storage, records are sorted by key. This will avoid unnecessary seeks.
+ Collections.sort(keys);
+
+ try (HoodieHFileReader<IndexedRecord> reader =
+ new HoodieHFileReader<>(inlineConf, inlinePath, new CacheConfig(inlineConf), inlinePath.getFileSystem(inlineConf))) {
+ // Get writer's schema from the header
+ List<Pair<String, IndexedRecord>> logRecords = reader.readRecords(keys, readerSchema);
+ return logRecords.stream().map(Pair::getSecond).collect(Collectors.toList());
}
- HoodieHFileReader reader = new HoodieHFileReader(inlineConf, inlinePath, cacheConf, inlinePath.getFileSystem(inlineConf));
- List<org.apache.hadoop.hbase.util.Pair<String, IndexedRecord>> logRecords = enableFullScan ? reader.readAllRecords(writerSchema, schema) :
- reader.readRecords(keys, schema);
- reader.close();
- this.records = logRecords.stream().map(t -> t.getSecond()).collect(Collectors.toList());
}
@Override
- protected void deserializeRecords() throws IOException {
+ protected List<IndexedRecord> deserializeRecords(byte[] content) throws IOException {
+ checkState(readerSchema != null, "Reader's schema has to be non-null");
Review comment:
Is this because of change in the PR? the original code seems to expect null reader schema at times?
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/ByteBufferBackedInputFile.java
##########
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hudi.common.util.io.ByteBufferBackedInputStream;
+import org.apache.parquet.io.DelegatingSeekableInputStream;
+import org.apache.parquet.io.InputFile;
+import org.apache.parquet.io.SeekableInputStream;
+
+/**
+ * Implementation of {@link InputFile} backed by {@code byte[]} buffer
+ */
+public class ByteBufferBackedInputFile implements InputFile {
+ private final byte[] buffer;
+ private final int offset;
+ private final int length;
+
+ public ByteBufferBackedInputFile(byte[] buffer, int offset, int length) {
+ this.buffer = buffer;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public ByteBufferBackedInputFile(byte[] buffer) {
+ this(buffer, 0, buffer.length);
+ }
+
+ @Override
+ public long getLength() {
+ return length;
+ }
+
+ @Override
+ public SeekableInputStream newStream() {
Review comment:
codecov was misleading. So we removed it. lets add unit tests for the new files.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
But let's fix this occurrence in this PR. Don't want it to be buried in a broader effort. Or all references you mention are around log reader/writer code - that can be prioritized right away?
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/OutputStreamBackedOutputFile.java
##########
@@ -0,0 +1,94 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.parquet.io.OutputFile;
+import org.apache.parquet.io.PositionOutputStream;
+
+import javax.annotation.Nonnull;
+import java.io.IOException;
+
+/**
+ * Implementation of the {@link OutputFile} backed by {@link java.io.OutputStream}
+ */
+public class OutputStreamBackedOutputFile implements OutputFile {
Review comment:
Do we need these to move away from older parquet APis? can you share more on why we are introducing the input/output file classes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006135812
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012726017
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012822806
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012717317
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012774985
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r840216398
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -144,79 +148,52 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected void createRecordsFromContentBytes() throws IOException {
- if (enableInlineReading) {
- getRecords(Collections.emptyList());
- } else {
- super.createRecordsFromContentBytes();
- }
- }
+ protected List<IndexedRecord> deserializeRecords(byte[] content) throws IOException {
+ checkState(readerSchema != null, "Reader's schema has to be non-null");
- @Override
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- readWithInlineFS(keys);
- return records;
- }
+ // Get schema from the header
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- /**
- * Serialize the record to byte buffer.
- *
- * @param record - Record to serialize
- * @param keyField - Key field in the schema
- * @return Serialized byte buffer for the record
- */
- private byte[] serializeRecord(final IndexedRecord record, final Option<Field> keyField) {
- if (keyField.isPresent()) {
- record.put(keyField.get().pos(), StringUtils.EMPTY_STRING);
- }
- return HoodieAvroUtils.indexedRecordToBytes(record);
+ // Read the content
+ HoodieHFileReader<IndexedRecord> reader = new HoodieHFileReader<>(content);
+ List<Pair<String, IndexedRecord>> records = reader.readAllRecords(writerSchema, readerSchema);
+
+ return records.stream().map(Pair::getSecond).collect(Collectors.toList());
}
- private void readWithInlineFS(List<String> keys) throws IOException {
- boolean enableFullScan = keys.isEmpty();
- // Get schema from the header
- Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- // If readerSchema was not present, use writerSchema
- if (schema == null) {
- schema = writerSchema;
- }
- Configuration conf = new Configuration();
- CacheConfig cacheConf = new CacheConfig(conf);
- Configuration inlineConf = new Configuration();
+ // TODO abstract this w/in HoodieDataBlock
+ @Override
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ // NOTE: It's important to extend Hadoop configuration here to make sure configuration
+ // is appropriately carried over
+ Configuration inlineConf = new Configuration(blockContentLoc.getHadoopConf());
Review comment:
wiring in hadoop conf from higher layers breaks this. Please check https://issues.apache.org/jira/browse/HUDI-3763
check L185 prior to this patch (where in inline was tested)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027406849
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027651306
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
}, {
"hash" : "29dd8404768aa1811844efc1332d02041971b8e0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "29dd8404768aa1811844efc1332d02041971b8e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
* 29dd8404768aa1811844efc1332d02041971b8e0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027735815
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
}, {
"hash" : "29dd8404768aa1811844efc1332d02041971b8e0",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676",
"triggerID" : "29dd8404768aa1811844efc1332d02041971b8e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 29dd8404768aa1811844efc1332d02041971b8e0 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r797123287
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,151 +18,173 @@
package org.apache.hudi.common.table.log.block;
-import org.apache.hudi.common.model.HoodieRecord;
-import org.apache.hudi.common.util.Option;
-import org.apache.hudi.exception.HoodieException;
-import org.apache.hudi.exception.HoodieIOException;
-
import org.apache.avro.Schema;
import org.apache.avro.generic.IndexedRecord;
import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private Option<List<IndexedRecord>> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
Review comment:
Double-checked this is indeed expected to just be a field-name (w/in the record's schema) not dot-path notation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012822806
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996323168
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r771646950
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -208,71 +207,85 @@ private HoodieLogBlock readBlock() throws IOException {
HoodieLogFormat.LogFormatVersion nextBlockVersion = readVersion();
// 3. Read the block type for a log block
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- type = inputStream.readInt();
-
- ValidationUtils.checkArgument(type < HoodieLogBlockType.values().length, "Invalid block byte type found " + type);
- blockType = HoodieLogBlockType.values()[type];
- }
+ HoodieLogBlockType blockType = tryReadBlockType(nextBlockVersion);
// 4. Read the header for a log block, if present
- if (nextBlockVersion.hasHeader()) {
- header = HoodieLogBlock.getLogMetadata(inputStream);
- }
- int contentLength = blocksize;
+ Map<HeaderMetadataType, String> header =
+ nextBlockVersion.hasHeader() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
+
// 5. Read the content length for the content
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- contentLength = (int) inputStream.readLong();
- }
+ // Fallback to full-block size if no content-length
+ // TODO replace w/ hasContentLength
+ int contentLength =
+ nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION ? (int) inputStream.readLong() : blockSize;
// 6. Read the content or skip content based on IO vs Memory trade-off by client
- // TODO - have a max block size and reuse this buffer in the ByteBuffer
- // (hard to guess max block size for now)
long contentPosition = inputStream.getPos();
- byte[] content = HoodieLogBlock.readOrSkipContent(inputStream, contentLength, readBlockLazily);
+ boolean shouldReadLazily = readBlockLazily && nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION;
Review comment:
B/c DEFAULT_VERSION could not be read lazily -- it would fail if we try to do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997129628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
* cecde3b6734576c5f2863ec2b4b90689600cb746 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996323168
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996230157
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996995275
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997033082
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998347129
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999335468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999873768
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
* 573af13b233334d2678ab40ca51c7424757aae60 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998320359
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998370268
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005424000
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005356347
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012716026
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012716026
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995352818
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995426398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
* 1a390d804e170c0fd42014211db65fd44a3d054e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995489753
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027406849
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997098222
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997098222
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996281613
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005448595
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005448595
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005336147
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
* be81e69fc9e51a211520de6e86efb7f7192a39db UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006174123
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012728429
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012728429
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012701103
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027483972
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995351561
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997118146
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996282083
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997118146
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997033082
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* e55e28fa924315ebc1b964f03440e3f3cd02732a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996205337
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r771044754
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,50 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
+ "Invalid InLineFS path: " + inlineFSPath);
final String pathExceptScheme = basePath.toString().substring(basePath.toString().indexOf(SCHEME_SEPARATOR) + 1);
- final String fullPath = scheme + SCHEME_SEPARATOR
- + (scheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ final String fullPath = baseFileScheme + SCHEME_SEPARATOR
+ + (baseFileScheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ pathExceptScheme;
return new Path(fullPath);
}
/**
- * Eg input : "inlinefs://file1/s3a/?start_offset=20&length=40".
- * output: 20
+ * Returns start offset w/in the base for the block identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs://file1/s3a/?start_offset=20&length=40".
+ * output: 20
*/
- public static int startOffset(Path inlinePath) {
- String[] slices = inlinePath.toString().split("[?&=]");
+ public static int startOffset(Path inlineFSPath) {
+ assertInlineFSPath(inlineFSPath);
+
+ String[] slices = inlineFSPath.toString().split("[?&=]");
return Integer.parseInt(slices[slices.length - 3]);
}
/**
- * Eg input : "inlinefs:/file1/s3a/?start_offset=20&length=40".
- * Output: 40
+ * Returns length of the block (embedded w/in the base file) identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs:/file1/s3a/?start_offset=20&length=40".
+ * output: 40
*/
public static int length(Path inlinePath) {
+ assertInlineFSPath(inlinePath);
Review comment:
same here.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +136,94 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
Review comment:
shouldn't the condition be AND instead of OR ?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -208,71 +207,85 @@ private HoodieLogBlock readBlock() throws IOException {
HoodieLogFormat.LogFormatVersion nextBlockVersion = readVersion();
// 3. Read the block type for a log block
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- type = inputStream.readInt();
-
- ValidationUtils.checkArgument(type < HoodieLogBlockType.values().length, "Invalid block byte type found " + type);
- blockType = HoodieLogBlockType.values()[type];
- }
+ HoodieLogBlockType blockType = tryReadBlockType(nextBlockVersion);
// 4. Read the header for a log block, if present
- if (nextBlockVersion.hasHeader()) {
- header = HoodieLogBlock.getLogMetadata(inputStream);
- }
- int contentLength = blocksize;
+ Map<HeaderMetadataType, String> header =
+ nextBlockVersion.hasHeader() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
+
// 5. Read the content length for the content
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- contentLength = (int) inputStream.readLong();
- }
+ // Fallback to full-block size if no content-length
+ // TODO replace w/ hasContentLength
+ int contentLength =
+ nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION ? (int) inputStream.readLong() : blockSize;
// 6. Read the content or skip content based on IO vs Memory trade-off by client
- // TODO - have a max block size and reuse this buffer in the ByteBuffer
- // (hard to guess max block size for now)
long contentPosition = inputStream.getPos();
- byte[] content = HoodieLogBlock.readOrSkipContent(inputStream, contentLength, readBlockLazily);
+ boolean shouldReadLazily = readBlockLazily && nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION;
Review comment:
why this change? previously we just had readBlockLazily if I am not wrong. can you throw some light.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
can we file a tracking ticket. we need to introduce configs for these.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,50 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
+ "Invalid InLineFS path: " + inlineFSPath);
final String pathExceptScheme = basePath.toString().substring(basePath.toString().indexOf(SCHEME_SEPARATOR) + 1);
- final String fullPath = scheme + SCHEME_SEPARATOR
- + (scheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ final String fullPath = baseFileScheme + SCHEME_SEPARATOR
+ + (baseFileScheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ pathExceptScheme;
return new Path(fullPath);
}
/**
- * Eg input : "inlinefs://file1/s3a/?start_offset=20&length=40".
- * output: 20
+ * Returns start offset w/in the base for the block identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs://file1/s3a/?start_offset=20&length=40".
+ * output: 20
*/
- public static int startOffset(Path inlinePath) {
- String[] slices = inlinePath.toString().split("[?&=]");
+ public static int startOffset(Path inlineFSPath) {
+ assertInlineFSPath(inlineFSPath);
Review comment:
do we need to assert for every such call ?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -36,50 +29,63 @@
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hudi.avro.HoodieAvroUtils;
+import org.apache.hudi.common.fs.SizeAwareDataInputStream;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
+import javax.annotation.Nonnull;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
-import javax.annotation.Nonnull;
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* HoodieAvroDataBlock contains a list of records serialized using Avro. It is used with the Parquet base file format.
*/
public class HoodieAvroDataBlock extends HoodieDataBlock {
- private ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
- private ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
-
- public HoodieAvroDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- }
-
- public HoodieAvroDataBlock(HoodieLogFile logFile, FSDataInputStream inputStream, Option<byte[]> content,
- boolean readBlockLazily, long position, long blockSize, long blockEndpos, Schema readerSchema,
- Map<HeaderMetadataType, String> header, Map<HeaderMetadataType, String> footer, String keyField) {
+ private final ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
+ private final ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
+
+ public HoodieAvroDataBlock(
+ HoodieLogFile logFile,
Review comment:
wondering if the code style has been applied correctly?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +136,94 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
Review comment:
read blocks lazily was present in this method prior to this patch right? was it removed intentionally ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998321696
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999327412
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
* 2f13b35851760cc2e62a702951041cfabb5453de UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999887877
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006137308
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005414443
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r778577147
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -208,71 +155,85 @@ private HoodieLogBlock readBlock() throws IOException {
HoodieLogFormat.LogFormatVersion nextBlockVersion = readVersion();
// 3. Read the block type for a log block
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- type = inputStream.readInt();
-
- ValidationUtils.checkArgument(type < HoodieLogBlockType.values().length, "Invalid block byte type found " + type);
- blockType = HoodieLogBlockType.values()[type];
- }
+ HoodieLogBlockType blockType = tryReadBlockType(nextBlockVersion);
// 4. Read the header for a log block, if present
- if (nextBlockVersion.hasHeader()) {
- header = HoodieLogBlock.getLogMetadata(inputStream);
- }
- int contentLength = blocksize;
+ Map<HeaderMetadataType, String> header =
+ nextBlockVersion.hasHeader() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
+
// 5. Read the content length for the content
- if (nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION) {
- contentLength = (int) inputStream.readLong();
- }
+ // Fallback to full-block size if no content-length
+ // TODO replace w/ hasContentLength
+ int contentLength =
+ nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION ? (int) inputStream.readLong() : blockSize;
// 6. Read the content or skip content based on IO vs Memory trade-off by client
- // TODO - have a max block size and reuse this buffer in the ByteBuffer
- // (hard to guess max block size for now)
long contentPosition = inputStream.getPos();
- byte[] content = HoodieLogBlock.readOrSkipContent(inputStream, contentLength, readBlockLazily);
+ boolean shouldReadLazily = readBlockLazily && nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION;
+ Option<byte[]> content = HoodieLogBlock.tryReadContent(inputStream, contentLength, shouldReadLazily);
// 7. Read footer if any
- Map<HeaderMetadataType, String> footer = null;
- if (nextBlockVersion.hasFooter()) {
- footer = HoodieLogBlock.getLogMetadata(inputStream);
- }
+ Map<HeaderMetadataType, String> footer =
+ nextBlockVersion.hasFooter() ? HoodieLogBlock.getLogMetadata(inputStream) : null;
// 8. Read log block length, if present. This acts as a reverse pointer when traversing a
// log file in reverse
- @SuppressWarnings("unused")
- long logBlockLength = 0;
if (nextBlockVersion.hasLogBlockLength()) {
- logBlockLength = inputStream.readLong();
+ inputStream.readLong();
}
// 9. Read the log block end position in the log file
long blockEndPos = inputStream.getPos();
switch (Objects.requireNonNull(blockType)) {
- // based on type read the block
case AVRO_DATA_BLOCK:
if (nextBlockVersion.getVersion() == HoodieLogFormatVersion.DEFAULT_VERSION) {
- return HoodieAvroDataBlock.getBlock(content, readerSchema);
+ return HoodieAvroDataBlock.getBlock(content.get(), readerSchema);
} else {
- return new HoodieAvroDataBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
- contentPosition, contentLength, blockEndPos, readerSchema, header, footer, keyField);
+ return new HoodieAvroDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema), header, footer, keyField);
}
+
case HFILE_DATA_BLOCK:
- return new HoodieHFileDataBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
- contentPosition, contentLength, blockEndPos, readerSchema,
- header, footer, enableInlineReading, keyField);
+ checkState(nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION,
+ String.format("HFile block could not be of version (%d)", HoodieLogFormatVersion.DEFAULT_VERSION));
+
+ return new HoodieHFileDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema),
+ header, footer, keyField, enableRecordLookups);
+
+ case PARQUET_DATA_BLOCK:
+ checkState(nextBlockVersion.getVersion() != HoodieLogFormatVersion.DEFAULT_VERSION,
+ String.format("Parquet block could not be of version (%d)", HoodieLogFormatVersion.DEFAULT_VERSION));
+
+ return new HoodieParquetDataBlock(logFile, inputStream, content, readBlockLazily,
+ contentPosition, contentLength, blockEndPos, Option.ofNullable(readerSchema), header, footer, keyField);
+
case DELETE_BLOCK:
- return HoodieDeleteBlock.getBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
+ return HoodieDeleteBlock.getBlock(logFile, inputStream, content, readBlockLazily,
contentPosition, contentLength, blockEndPos, header, footer);
+
case COMMAND_BLOCK:
- return HoodieCommandBlock.getBlock(logFile, inputStream, Option.ofNullable(content), readBlockLazily,
+ return HoodieCommandBlock.getBlock(logFile, inputStream, content, readBlockLazily,
contentPosition, contentLength, blockEndPos, header, footer);
+
default:
throw new HoodieNotSupportedException("Unsupported Block " + blockType);
}
}
+ @Nullable
+ private HoodieLogBlockType tryReadBlockType(HoodieLogFormat.LogFormatVersion blockVersion) throws IOException {
+ if (blockVersion.getVersion() == HoodieLogFormatVersion.DEFAULT_VERSION) {
+ return null;
Review comment:
While addressing this i've stumbled upon peculiar behavior that is currently implemented: for `DEFAULT_VERSION` block reading would actually fail since it doesn't contain block-type, and we assert at line 252 (before change) that it's not null.
While this doesn't seem right, i don't thin i have a good idea around what should be the behavior since we can't reliably determine what block is in-front of us and it seems that DEFAULT_VERSION supported Avro, Delete, Corrupt blocks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006135812
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006174123
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005336147
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
* be81e69fc9e51a211520de6e86efb7f7192a39db UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006137308
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006146863
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r797120652
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -224,6 +217,33 @@ public static HoodieAvroDataBlock getBlock(byte[] content, Schema readerSchema)
return new HoodieAvroDataBlock(records, readerSchema);
}
+ private static byte[] compress(String text) {
Review comment:
This just have been moved down to make it easier to analyze actually overridden methods.
> can we move this to IOUtils or some common utils class?
This are actually a deprecated methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027384679
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027682215
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
}, {
"hash" : "29dd8404768aa1811844efc1332d02041971b8e0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676",
"triggerID" : "29dd8404768aa1811844efc1332d02041971b8e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
* 29dd8404768aa1811844efc1332d02041971b8e0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995352818
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r771648072
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -36,50 +29,63 @@
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hudi.avro.HoodieAvroUtils;
+import org.apache.hudi.common.fs.SizeAwareDataInputStream;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
+import javax.annotation.Nonnull;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
-import javax.annotation.Nonnull;
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* HoodieAvroDataBlock contains a list of records serialized using Avro. It is used with the Parquet base file format.
*/
public class HoodieAvroDataBlock extends HoodieDataBlock {
- private ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
- private ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
-
- public HoodieAvroDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- }
-
- public HoodieAvroDataBlock(HoodieLogFile logFile, FSDataInputStream inputStream, Option<byte[]> content,
- boolean readBlockLazily, long position, long blockSize, long blockEndpos, Schema readerSchema,
- Map<HeaderMetadataType, String> header, Map<HeaderMetadataType, String> footer, String keyField) {
+ private final ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
+ private final ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
+
+ public HoodieAvroDataBlock(
+ HoodieLogFile logFile,
Review comment:
Can you elaborate what aspect you are referring to? Stacking or tabbing?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,50 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
+ "Invalid InLineFS path: " + inlineFSPath);
final String pathExceptScheme = basePath.toString().substring(basePath.toString().indexOf(SCHEME_SEPARATOR) + 1);
- final String fullPath = scheme + SCHEME_SEPARATOR
- + (scheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ final String fullPath = baseFileScheme + SCHEME_SEPARATOR
+ + (baseFileScheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ pathExceptScheme;
return new Path(fullPath);
}
/**
- * Eg input : "inlinefs://file1/s3a/?start_offset=20&length=40".
- * output: 20
+ * Returns start offset w/in the base for the block identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs://file1/s3a/?start_offset=20&length=40".
+ * output: 20
*/
- public static int startOffset(Path inlinePath) {
- String[] slices = inlinePath.toString().split("[?&=]");
+ public static int startOffset(Path inlineFSPath) {
+ assertInlineFSPath(inlineFSPath);
+
+ String[] slices = inlineFSPath.toString().split("[?&=]");
return Integer.parseInt(slices[slices.length - 3]);
}
/**
- * Eg input : "inlinefs:/file1/s3a/?start_offset=20&length=40".
- * Output: 40
+ * Returns length of the block (embedded w/in the base file) identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs:/file1/s3a/?start_offset=20&length=40".
+ * output: 40
*/
public static int length(Path inlinePath) {
+ assertInlineFSPath(inlinePath);
Review comment:
Responded above
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +136,94 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
Review comment:
We are either reading it from the log (in which case we have content on read-path) or write them into it (in which case we have records, on the write-path)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997131996
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996987870
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
* e55e28fa924315ebc1b964f03440e3f3cd02732a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r771646123
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,50 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
+ "Invalid InLineFS path: " + inlineFSPath);
final String pathExceptScheme = basePath.toString().substring(basePath.toString().indexOf(SCHEME_SEPARATOR) + 1);
- final String fullPath = scheme + SCHEME_SEPARATOR
- + (scheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ final String fullPath = baseFileScheme + SCHEME_SEPARATOR
+ + (baseFileScheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ pathExceptScheme;
return new Path(fullPath);
}
/**
- * Eg input : "inlinefs://file1/s3a/?start_offset=20&length=40".
- * output: 20
+ * Returns start offset w/in the base for the block identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs://file1/s3a/?start_offset=20&length=40".
+ * output: 20
*/
- public static int startOffset(Path inlinePath) {
- String[] slices = inlinePath.toString().split("[?&=]");
+ public static int startOffset(Path inlineFSPath) {
+ assertInlineFSPath(inlineFSPath);
Review comment:
Assertion would provide clear failure point, and it's a cheap check to make (we just compare schemes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r772709134
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
Review comment:
Actually the one below will do
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
https://issues.apache.org/jira/browse/HUDI-3055
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
There are quite a few places where this occurs, we need to do a deeper scan to fix all of this occurrences
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998321696
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-998320359
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997144563
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999365421
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r772667199
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
inlineConf is just getting instantiated at L153 and not wired in from higher layer.Just wanted to remind just incase we might miss anything as this conf is just empty except for file system scheme impl.
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/OutputStreamBackedOutputFile.java
##########
@@ -0,0 +1,94 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.parquet.io.OutputFile;
+import org.apache.parquet.io.PositionOutputStream;
+
+import javax.annotation.Nonnull;
+import java.io.IOException;
+
+/**
+ * Implementation of the {@link OutputFile} backed by {@link java.io.OutputStream}
+ */
+public class OutputStreamBackedOutputFile implements OutputFile {
Review comment:
Are these inspired from somewhere? I see StreamOutputFile in flink-parquet library which resembles this.
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/ByteBufferBackedInputFile.java
##########
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hudi.common.util.io.ByteBufferBackedInputStream;
+import org.apache.parquet.io.DelegatingSeekableInputStream;
+import org.apache.parquet.io.InputFile;
+import org.apache.parquet.io.SeekableInputStream;
+
+/**
+ * Implementation of {@link InputFile} backed by {@code byte[]} buffer
+ */
+public class ByteBufferBackedInputFile implements InputFile {
+ private final byte[] buffer;
+ private final int offset;
+ private final int length;
+
+ public ByteBufferBackedInputFile(byte[] buffer, int offset, int length) {
+ this.buffer = buffer;
+ this.offset = offset;
+ this.length = length;
+ }
+
+ public ByteBufferBackedInputFile(byte[] buffer) {
+ this(buffer, 0, buffer.length);
+ }
+
+ @Override
+ public long getLength() {
+ return length;
+ }
+
+ @Override
+ public SeekableInputStream newStream() {
Review comment:
I see lot of new files here. do we have tests for these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r773607983
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,49 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
Review comment:
Can we stick to inner/outer terminology? given "base file" a very different meaning in Hudi
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +132,97 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records);
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
if (records == null) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = readRecordsFromBlockPayload();
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
return records;
}
+ public Schema getSchema() {
+ return readerSchema;
+ }
+
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
Review comment:
note to self ; this should not affect the returned value, since that's a new list
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -125,6 +126,11 @@
.withAlternatives("hoodie.table.rt.file.format")
.withDocumentation("Log format used for the delta logs.");
+ public static final ConfigProperty<String> LOG_BLOCK_TYPE = ConfigProperty
Review comment:
I would love for this to be dynamically figured out based on the log block header? We anyway read them out first before processing the blocks themselves. This way a streaming writer can be ingesting avro blocks, while a background delete/backfill can be adding columnar data into the log blocks. Let's remove the table config?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,59 +18,79 @@
package org.apache.hudi.common.table.log.block;
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hudi.common.model.HoodieRecord;
import org.apache.hudi.common.util.Option;
import org.apache.hudi.exception.HoodieException;
import org.apache.hudi.exception.HoodieIOException;
-import org.apache.avro.Schema;
-import org.apache.avro.generic.IndexedRecord;
-import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
-
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private List<IndexedRecord> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
Review comment:
rename: keyFieldName to be consistent
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -224,6 +226,33 @@ public static HoodieAvroDataBlock getBlock(byte[] content, Schema readerSchema)
return new HoodieAvroDataBlock(records, readerSchema);
}
+ private static byte[] compress(String text) {
Review comment:
Can we move this to StringUtils
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -157,17 +161,17 @@ protected void deserializeRecords() throws IOException {
// 3. Read the content
for (int i = 0; i < totalRecords; i++) {
int recordLength = dis.readInt();
- BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(getContent().get(), dis.getNumberOfBytesRead(),
+ BinaryDecoder decoder = DecoderFactory.get().binaryDecoder(content, dis.getNumberOfBytesRead(),
recordLength, decoderCache.get());
decoderCache.set(decoder);
IndexedRecord record = reader.read(null, decoder);
records.add(record);
dis.skipBytes(recordLength);
}
+
dis.close();
- this.records = records;
- // Free up content to be GC'd, deflate
- deflate();
Review comment:
IIUC since we just pass records in now and not hang onto it, we are good. This is a nice cleanup!
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,59 +18,79 @@
package org.apache.hudi.common.table.log.block;
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hudi.common.model.HoodieRecord;
import org.apache.hudi.common.util.Option;
import org.apache.hudi.exception.HoodieException;
import org.apache.hudi.exception.HoodieIOException;
-import org.apache.avro.Schema;
-import org.apache.avro.generic.IndexedRecord;
-import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
-
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private List<IndexedRecord> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
+
+ private final boolean enablePointLookups;
- public HoodieDataBlock(@Nonnull List<IndexedRecord> records, @Nonnull Map<HeaderMetadataType, String> header,
- @Nonnull Map<HeaderMetadataType, String> footer, String keyField) {
- this(header, footer, Option.empty(), Option.empty(), null, false);
+ protected final Schema readerSchema;
+
+ /**
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
+ */
+ public HoodieDataBlock(List<IndexedRecord> records,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef) {
+ super(header, footer, Option.empty(), Option.empty(), null, false);
this.records = records;
- this.schema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- this.keyField = keyField;
+ this.keyFieldRef = keyFieldRef;
+ // If no reader-schema has been provided assume writer-schema as one
+ this.readerSchema = getWriterSchema(super.getLogBlockHeader());
+ this.enablePointLookups = false;
}
- protected HoodieDataBlock(Option<byte[]> content, @Nonnull FSDataInputStream inputStream, boolean readBlockLazily,
- Option<HoodieLogBlockContentLocation> blockContentLocation, Schema readerSchema,
- @Nonnull Map<HeaderMetadataType, String> headers, @Nonnull Map<HeaderMetadataType,
- String> footer, String keyField) {
- this(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
- this.schema = readerSchema;
- this.keyField = keyField;
+ /**
+ * NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
+ */
+ protected HoodieDataBlock(Option<byte[]> content,
+ FSDataInputStream inputStream,
+ boolean readBlockLazily,
+ Option<HoodieLogBlockContentLocation> blockContentLocation,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> headers,
+ Map<HeaderMetadataType, String> footer,
+ String keyFieldRef,
+ boolean enablePointLookups) {
+ super(headers, footer, blockContentLocation, content, inputStream, readBlockLazily);
+ this.records = null;
Review comment:
Can we use `Option` instead of `null`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,49 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
Review comment:
please unwrap this line (and similar everywhere) to keep things consistent with how it is?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -36,50 +29,67 @@
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hudi.common.fs.SizeAwareDataInputStream;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
+import javax.annotation.Nonnull;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
+import java.util.zip.DeflaterOutputStream;
+import java.util.zip.InflaterInputStream;
-import javax.annotation.Nonnull;
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* HoodieAvroDataBlock contains a list of records serialized using Avro. It is used with the Parquet base file format.
*/
public class HoodieAvroDataBlock extends HoodieDataBlock {
- private ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
- private ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
-
- public HoodieAvroDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- }
-
- public HoodieAvroDataBlock(HoodieLogFile logFile, FSDataInputStream inputStream, Option<byte[]> content,
- boolean readBlockLazily, long position, long blockSize, long blockEndpos, Schema readerSchema,
- Map<HeaderMetadataType, String> header, Map<HeaderMetadataType, String> footer, String keyField) {
+ private final ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
+ private final ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
+
+ public HoodieAvroDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
super(content, inputStream, readBlockLazily,
- Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)), readerSchema, header,
- footer, keyField);
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)), readerSchema, header,
+ footer, keyField, false);
}
- public HoodieAvroDataBlock(@Nonnull List<IndexedRecord> records, @Nonnull Map<HeaderMetadataType,
- String> header, String keyField) {
+ public HoodieAvroDataBlock(
Review comment:
Can we please revert this
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +132,97 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records);
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
if (records == null) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = readRecordsFromBlockPayload();
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
return records;
}
+ public Schema getSchema() {
+ return readerSchema;
+ }
+
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
+ }
+ }
+
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ throw new UnsupportedOperationException(
+ String.format("Point-wise records lookups are not supported by this Data block type (%s)", getBlockType())
Review comment:
nit: point lookups are not ...
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java
##########
@@ -93,43 +105,48 @@ public HoodieLogBlockType getBlockType() {
}
@Override
- protected byte[] serializeRecords() throws IOException {
- HFileContext context = new HFileContextBuilder().withBlockSize(blockSize).withCompression(compressionAlgorithm)
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ HFileContext context = new HFileContextBuilder()
+ .withBlockSize(DEFAULT_BLOCK_SIZE)
+ // TODO fetch value from the config
+ .withCompression(DEFAULT_COMPRESSION_ALGO)
Review comment:
can we fetch from the HFile config in this PR itself?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -36,50 +29,63 @@
import org.apache.avro.io.Encoder;
import org.apache.avro.io.EncoderFactory;
import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hudi.avro.HoodieAvroUtils;
+import org.apache.hudi.common.fs.SizeAwareDataInputStream;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.exception.HoodieIOException;
+import javax.annotation.Nonnull;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
-import javax.annotation.Nonnull;
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* HoodieAvroDataBlock contains a list of records serialized using Avro. It is used with the Parquet base file format.
*/
public class HoodieAvroDataBlock extends HoodieDataBlock {
- private ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
- private ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
-
- public HoodieAvroDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- }
-
- public HoodieAvroDataBlock(HoodieLogFile logFile, FSDataInputStream inputStream, Option<byte[]> content,
- boolean readBlockLazily, long position, long blockSize, long blockEndpos, Schema readerSchema,
- Map<HeaderMetadataType, String> header, Map<HeaderMetadataType, String> footer, String keyField) {
+ private final ThreadLocal<BinaryEncoder> encoderCache = new ThreadLocal<>();
+ private final ThreadLocal<BinaryDecoder> decoderCache = new ThreadLocal<>();
+
+ public HoodieAvroDataBlock(
+ HoodieLogFile logFile,
Review comment:
Can we please stick to how the current methods are formatted along the arguments? wrapped and indented after the method's `(`.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +132,97 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records);
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
if (records == null) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = readRecordsFromBlockPayload();
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
return records;
}
+ public Schema getSchema() {
+ return readerSchema;
+ }
+
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
+ }
+ }
+
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ throw new UnsupportedOperationException(
+ String.format("Point-wise records lookups are not supported by this Data block type (%s)", getBlockType())
+ );
}
- protected abstract byte[] serializeRecords() throws IOException;
+ protected abstract byte[] serializeRecords(List<IndexedRecord> records) throws IOException;
+
+ protected abstract List<IndexedRecord> deserializeRecords(byte[] content) throws IOException;
- protected abstract void deserializeRecords() throws IOException;
+ public abstract HoodieLogBlockType getBlockType();
+
+ protected String getRecordKey(IndexedRecord record) {
Review comment:
wondering if there is a reusable helper for this. how would this work for different key generators ? we seem to assume there is a single key field - which is true always when hudi metadata is enabled. something to think about.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006099093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012727208
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012786245
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-995366619
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7bce55782061189c539ae7acb1e9b87ade7cdf7f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345)
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 00df74e3e4f6039008cd26b789218b48e6ea0d6a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996238336
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996230157
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* 1a390d804e170c0fd42014211db65fd44a3d054e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363)
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-997131996
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 67cbb2f4ab421fb7a90e4c5d1061613ed331c837 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460)
* cecde3b6734576c5f2863ec2b4b90689600cb746 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r772083398
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
Review comment:
this will do the projection and the columnar reads as discussed?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
We can take values from writeConfig > parquetConfig or sth.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/fs/inline/InLineFSUtils.java
##########
@@ -74,40 +75,50 @@ public static Path getInlineFilePath(Path outerPath, String origScheme, long inL
* @return Outer file Path from the InLineFS Path
*/
public static Path getOuterFilePathFromInlinePath(Path inlineFSPath) {
- final String scheme = inlineFSPath.getParent().getName();
+ assertInlineFSPath(inlineFSPath);
+
+ final String baseFileScheme = inlineFSPath.getParent().getName();
final Path basePath = inlineFSPath.getParent().getParent();
- ValidationUtils.checkArgument(basePath.toString().contains(SCHEME_SEPARATOR),
- "Invalid InLineFSPath: " + inlineFSPath);
+ checkArgument(
+ basePath.toString().contains(SCHEME_SEPARATOR),
+ "Invalid InLineFS path: " + inlineFSPath);
final String pathExceptScheme = basePath.toString().substring(basePath.toString().indexOf(SCHEME_SEPARATOR) + 1);
- final String fullPath = scheme + SCHEME_SEPARATOR
- + (scheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ final String fullPath = baseFileScheme + SCHEME_SEPARATOR
+ + (baseFileScheme.equals(LOCAL_FILESYSTEM_SCHEME) ? PATH_SEPARATOR : "")
+ pathExceptScheme;
return new Path(fullPath);
}
/**
- * Eg input : "inlinefs://file1/s3a/?start_offset=20&length=40".
- * output: 20
+ * Returns start offset w/in the base for the block identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs://file1/s3a/?start_offset=20&length=40".
+ * output: 20
*/
- public static int startOffset(Path inlinePath) {
- String[] slices = inlinePath.toString().split("[?&=]");
+ public static int startOffset(Path inlineFSPath) {
+ assertInlineFSPath(inlineFSPath);
+
+ String[] slices = inlineFSPath.toString().split("[?&=]");
return Integer.parseInt(slices[slices.length - 3]);
}
/**
- * Eg input : "inlinefs:/file1/s3a/?start_offset=20&length=40".
- * Output: 40
+ * Returns length of the block (embedded w/in the base file) identified by the given InlineFS path
*
- * @param inlinePath
- * @return
+ * input: "inlinefs:/file1/s3a/?start_offset=20&length=40".
+ * output: 40
*/
public static int length(Path inlinePath) {
+ assertInlineFSPath(inlinePath);
+
String[] slices = inlinePath.toString().split("[?&=]");
return Integer.parseInt(slices[slices.length - 1]);
}
+ private static void assertInlineFSPath(Path inlinePath) {
+ String scheme = inlinePath.toUri().getScheme();
+
Review comment:
nit: remove empty line?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
can we just fix it in this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r778468264
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +136,94 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
Review comment:
Correct. This could should not be reachable (this method is only used on the write path)
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.fs.inline.InLineFSUtils;
+import org.apache.hudi.common.fs.inline.InLineFileSystem;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.HadoopInputFile;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily,
+ long position, long blockSize, long blockEndPos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField) {
+ super(content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndPos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
+ ParquetWriter.DEFAULT_BLOCK_SIZE,
+ ParquetWriter.DEFAULT_PAGE_SIZE,
+ 1024 * 1024 * 1024,
+ new Configuration(),
+ Double.parseDouble(String.valueOf(0.1)));//HoodieStorageConfig.PARQUET_COMPRESSION_RATIO.defaultValue()));
+
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+
+ try (FSDataOutputStream outputStream = new FSDataOutputStream(baos)) {
+ try (HoodieParquetStreamWriter<IndexedRecord> parquetWriter = new HoodieParquetStreamWriter<>(outputStream, avroParquetConfig)) {
+ for (IndexedRecord record : records) {
+ String recordKey = getRecordKey(record);
+ parquetWriter.writeAvro(recordKey, record);
+ }
+ outputStream.flush();
+ }
+ }
+
+ return baos.toByteArray();
+ }
+
+ public static Iterator<IndexedRecord> getProjectedParquetRecordsIterator(Configuration conf,
+ Schema readerSchema,
+ InputFile inputFile) throws IOException {
+ AvroReadSupport.setAvroReadSchema(conf, readerSchema);
+ AvroReadSupport.setRequestedProjection(conf, readerSchema);
+ ParquetReader<IndexedRecord> reader =
+ AvroParquetReader.<IndexedRecord>builder(inputFile).withConf(conf).build();
+ return new ParquetReaderIterator<>(reader);
+ }
+
+ /**
+ * NOTE: We're overriding the whole reading sequence to make sure we properly respect
+ * the requested Reader's schema and only fetch the columns that have been explicitly
+ * requested by the caller (providing projected Reader's schema)
+ */
+ @Override
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
+ Configuration inlineConf = new Configuration();
+ inlineConf.set("fs." + InLineFileSystem.SCHEME + ".impl", InLineFileSystem.class.getName());
+
+ HoodieLogBlockContentLocation blockContentLoc = getBlockContentLocation().get();
+
+ Path inlineLogFilePath = InLineFSUtils.getInlineFilePath(
+ blockContentLoc.getLogFile().getPath(),
+ blockContentLoc.getLogFile().getPath().getFileSystem(inlineConf).getScheme(),
+ blockContentLoc.getContentPositionInLogFile(),
+ blockContentLoc.getBlockSize());
+
+ ArrayList<IndexedRecord> records = new ArrayList<>();
+
+ getProjectedParquetRecordsIterator(
+ inlineConf,
Review comment:
Makes sense
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -18,59 +18,79 @@
package org.apache.hudi.common.table.log.block;
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hudi.common.model.HoodieRecord;
import org.apache.hudi.common.util.Option;
import org.apache.hudi.exception.HoodieException;
import org.apache.hudi.exception.HoodieIOException;
-import org.apache.avro.Schema;
-import org.apache.avro.generic.IndexedRecord;
-import org.apache.hadoop.fs.FSDataInputStream;
-
-import javax.annotation.Nonnull;
-
import java.io.IOException;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.util.ValidationUtils.checkState;
/**
* DataBlock contains a list of records serialized using formats compatible with the base file format.
* For each base file format there is a corresponding DataBlock format.
- *
+ * <p>
* The Datablock contains:
* 1. Data Block version
* 2. Total number of records in the block
* 3. Actual serialized content of the records
*/
public abstract class HoodieDataBlock extends HoodieLogBlock {
- protected List<IndexedRecord> records;
- protected Schema schema;
- protected String keyField;
+ // TODO rebase records/content to leverage Either to warrant
+ // that they are mutex (used by read/write flows respectively)
+ private List<IndexedRecord> records;
- public HoodieDataBlock(@Nonnull Map<HeaderMetadataType, String> logBlockHeader,
- @Nonnull Map<HeaderMetadataType, String> logBlockFooter,
- @Nonnull Option<HoodieLogBlockContentLocation> blockContentLocation, @Nonnull Option<byte[]> content,
- FSDataInputStream inputStream, boolean readBlockLazily) {
- super(logBlockHeader, logBlockFooter, blockContentLocation, content, inputStream, readBlockLazily);
- this.keyField = HoodieRecord.RECORD_KEY_METADATA_FIELD;
- }
+ /**
+ * Dot-path notation reference to the key field w/in the record's schema
+ */
+ private final String keyFieldRef;
Review comment:
`keyFieldName` is confusing -- it's not really a field name but rather FQ ref w/in the schema
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlock.java
##########
@@ -109,7 +116,25 @@ public long getLogBlockLength() {
* Type of the log block WARNING: This enum is serialized as the ordinal. Only add new enums at the end.
*/
public enum HoodieLogBlockType {
- COMMAND_BLOCK, DELETE_BLOCK, CORRUPT_BLOCK, AVRO_DATA_BLOCK, HFILE_DATA_BLOCK
+ COMMAND_BLOCK(":command"),
+ DELETE_BLOCK(":delete"),
+ CORRUPT_BLOCK(":corrupted"),
Review comment:
Corrupt is a valid adjective, but it's rarely used in a context of "broken" things as far as i can tell (more often used to describe lack of moral integrity like "politician is corrupt")
https://english.stackexchange.com/questions/405182/corrupt-or-corrupted
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieAvroDataBlock.java
##########
@@ -224,6 +226,33 @@ public static HoodieAvroDataBlock getBlock(byte[] content, Schema readerSchema)
return new HoodieAvroDataBlock(records, readerSchema);
}
+ private static byte[] compress(String text) {
Review comment:
This is only used in `HoodieAvroDataBlock` w/in the method that is deprecated. Do we want to hang on to it?
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieParquetDataBlock.java
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table.log.block;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataInputStream;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.common.util.ParquetReaderIterator;
+import org.apache.hudi.io.storage.HoodieAvroParquetConfig;
+import org.apache.hudi.io.storage.HoodieParquetStreamWriter;
+import org.apache.hudi.parquet.io.ByteBufferBackedInputFile;
+import org.apache.parquet.avro.AvroParquetReader;
+import org.apache.parquet.avro.AvroReadSupport;
+import org.apache.parquet.avro.AvroSchemaConverter;
+import org.apache.parquet.hadoop.ParquetReader;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.io.InputFile;
+
+import javax.annotation.Nonnull;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * HoodieParquetDataBlock contains a list of records serialized using Parquet.
+ */
+public class HoodieParquetDataBlock extends HoodieDataBlock {
+
+ public HoodieParquetDataBlock(
+ HoodieLogFile logFile,
+ FSDataInputStream inputStream,
+ Option<byte[]> content,
+ boolean readBlockLazily, long position, long blockSize, long blockEndpos,
+ Option<Schema> readerSchema,
+ Map<HeaderMetadataType, String> header,
+ Map<HeaderMetadataType, String> footer,
+ String keyField
+ ) {
+ super(
+ content,
+ inputStream,
+ readBlockLazily,
+ Option.of(new HoodieLogBlockContentLocation(logFile, position, blockSize, blockEndpos)),
+ readerSchema,
+ header,
+ footer,
+ keyField,
+ false);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header,
+ @Nonnull String keyField
+ ) {
+ super(records, header, new HashMap<>(), keyField);
+ }
+
+ public HoodieParquetDataBlock(
+ @Nonnull List<IndexedRecord> records,
+ @Nonnull Map<HeaderMetadataType, String> header
+ ) {
+ super(records, header, new HashMap<>(), HoodieRecord.RECORD_KEY_METADATA_FIELD);
+ }
+
+ @Override
+ public HoodieLogBlockType getBlockType() {
+ return HoodieLogBlockType.PARQUET_DATA_BLOCK;
+ }
+
+ @Override
+ protected byte[] serializeRecords(List<IndexedRecord> records) throws IOException {
+ if (records.size() == 0) {
+ return new byte[0];
+ }
+
+ Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
+
+ HoodieAvroWriteSupport writeSupport = new HoodieAvroWriteSupport(
+ new AvroSchemaConverter().convert(writerSchema), writerSchema, Option.empty());
+
+ HoodieAvroParquetConfig avroParquetConfig =
+ new HoodieAvroParquetConfig(
+ writeSupport,
+ // TODO fetch compression codec from the config
+ CompressionCodecName.GZIP,
Review comment:
It's a fair call given that this is a new code.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -125,6 +126,11 @@
.withAlternatives("hoodie.table.rt.file.format")
.withDocumentation("Log format used for the delta logs.");
+ public static final ConfigProperty<String> LOG_BLOCK_TYPE = ConfigProperty
Review comment:
How would this work? How's block-type going to be determined on the write path?
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieHFileReader.java
##########
@@ -288,28 +288,14 @@ public synchronized void close() {
}
}
- static class SeekableByteArrayInputStream extends ByteArrayInputStream implements Seekable, PositionedReadable {
+ static class SeekableByteArrayInputStream extends ByteBufferBackedInputStream implements Seekable, PositionedReadable {
Review comment:
It's not changing, just abstracting common functionality required elsewhere
##########
File path: hudi-common/src/main/java/org/apache/hudi/parquet/io/OutputStreamBackedOutputFile.java
##########
@@ -0,0 +1,94 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.parquet.io;
+
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.parquet.io.OutputFile;
+import org.apache.parquet.io.PositionOutputStream;
+
+import javax.annotation.Nonnull;
+import java.io.IOException;
+
+/**
+ * Implementation of the {@link OutputFile} backed by {@link java.io.OutputStream}
+ */
+public class OutputStreamBackedOutputFile implements OutputFile {
Review comment:
Correct. These are needed to be able to write Parquet into `OutputStream`
##########
File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieParquetStreamWriter.java
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io.storage;
+
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FSDataOutputStream;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.avro.HoodieAvroWriteSupport;
+import org.apache.hudi.parquet.io.OutputStreamBackedOutputFile;
+import org.apache.parquet.hadoop.ParquetFileWriter;
+import org.apache.parquet.hadoop.ParquetWriter;
+import org.apache.parquet.hadoop.api.WriteSupport;
+import org.apache.parquet.io.OutputFile;
+
+import java.io.IOException;
+
+// TODO unify w/ HoodieParquetWriter
+public class HoodieParquetStreamWriter<R extends IndexedRecord> implements AutoCloseable {
Review comment:
`ParquetWriter` ctors that are needed here are package-private and hence i can't extend `HoodieParquetWriter` to use them -- hence i had to wrap around `ParquetWriter` in here to be able to write into `OutputStream`
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieLogBlock.java
##########
@@ -109,7 +116,25 @@ public long getLogBlockLength() {
* Type of the log block WARNING: This enum is serialized as the ordinal. Only add new enums at the end.
*/
public enum HoodieLogBlockType {
- COMMAND_BLOCK, DELETE_BLOCK, CORRUPT_BLOCK, AVRO_DATA_BLOCK, HFILE_DATA_BLOCK
+ COMMAND_BLOCK(":command"),
Review comment:
To distinguish b/w format-based blocks and action-based blocks
##########
File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HadoopMapRedUtils.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.testutils;
+
+import org.apache.hadoop.mapred.Counters;
+import org.apache.hadoop.mapred.InputSplit;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hudi.common.util.Option;
+
+import java.util.concurrent.ConcurrentHashMap;
+
+public class HadoopMapRedUtils {
Review comment:
Fair call, but this is the only way Parquet reader provides insights into how much was actually read. If we don't want to add those we will have to forego these assertions.
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java
##########
@@ -110,59 +132,97 @@ public static HoodieLogBlock getBlock(HoodieLogBlockType logDataBlockFormat, Lis
@Override
public byte[] getContentBytes() throws IOException {
// In case this method is called before realizing records from content
- if (getContent().isPresent()) {
- return getContent().get();
- } else if (readBlockLazily && !getContent().isPresent() && records == null) {
- // read block lazily
- createRecordsFromContentBytes();
+ Option<byte[]> content = getContent();
+
+ checkState(content.isPresent() || records != null, "Block is in invalid state");
+
+ if (content.isPresent()) {
+ return content.get();
}
- return serializeRecords();
+ return serializeRecords(records);
}
- public abstract HoodieLogBlockType getBlockType();
+ protected static Schema getWriterSchema(Map<HeaderMetadataType, String> logBlockHeader) {
+ return new Schema.Parser().parse(logBlockHeader.get(HeaderMetadataType.SCHEMA));
+ }
- public List<IndexedRecord> getRecords() {
+ /**
+ * Returns all the records contained w/in this block
+ */
+ public final List<IndexedRecord> getRecords() {
if (records == null) {
try {
// in case records are absent, read content lazily and then convert to IndexedRecords
- createRecordsFromContentBytes();
+ records = readRecordsFromBlockPayload();
} catch (IOException io) {
throw new HoodieIOException("Unable to convert content bytes to records", io);
}
}
return records;
}
+ public Schema getSchema() {
+ return readerSchema;
+ }
+
/**
* Batch get of keys of interest. Implementation can choose to either do full scan and return matched entries or
* do a seek based parsing and return matched entries.
+ *
* @param keys keys of interest.
* @return List of IndexedRecords for the keys of interest.
- * @throws IOException
+ * @throws IOException in case of failures encountered when reading/parsing records
*/
- public List<IndexedRecord> getRecords(List<String> keys) throws IOException {
- throw new UnsupportedOperationException("On demand batch get based on interested keys not supported");
- }
+ public final List<IndexedRecord> getRecords(List<String> keys) throws IOException {
+ boolean fullScan = keys.isEmpty();
+ if (enablePointLookups && !fullScan) {
+ return lookupRecords(keys);
+ }
- public Schema getSchema() {
- // if getSchema was invoked before converting byte [] to records
- if (records == null) {
- getRecords();
+ // Otherwise, we fetch all the records and filter out all the records, but the
+ // ones requested
+ List<IndexedRecord> allRecords = getRecords();
+ if (fullScan) {
+ return allRecords;
}
- return schema;
+
+ HashSet<String> keySet = new HashSet<>(keys);
+ return allRecords.stream()
+ .filter(record -> keySet.contains(getRecordKey(record)))
+ .collect(Collectors.toList());
}
- protected void createRecordsFromContentBytes() throws IOException {
+ protected List<IndexedRecord> readRecordsFromBlockPayload() throws IOException {
if (readBlockLazily && !getContent().isPresent()) {
// read log block contents from disk
inflate();
}
- deserializeRecords();
+ try {
+ return deserializeRecords(getContent().get());
+ } finally {
+ // Free up content to be GC'd by deflating the block
+ deflate();
+ }
+ }
+
+ protected List<IndexedRecord> lookupRecords(List<String> keys) throws IOException {
+ throw new UnsupportedOperationException(
+ String.format("Point-wise records lookups are not supported by this Data block type (%s)", getBlockType())
+ );
}
- protected abstract byte[] serializeRecords() throws IOException;
+ protected abstract byte[] serializeRecords(List<IndexedRecord> records) throws IOException;
+
+ protected abstract List<IndexedRecord> deserializeRecords(byte[] content) throws IOException;
- protected abstract void deserializeRecords() throws IOException;
+ public abstract HoodieLogBlockType getBlockType();
+
+ protected String getRecordKey(IndexedRecord record) {
Review comment:
This is just an abstraction of the existing behavior: currently blocks don't support virtual keys, and if we want to do so i'd rather do this in a standalone PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999917098
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 573af13b233334d2678ab40ca51c7424757aae60 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1005424000
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* be81e69fc9e51a211520de6e86efb7f7192a39db Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894)
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1006113171
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 463e9f1675c105a65d1eeb26bf05d2ef06c4216c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897)
* 8b7962db7e7697450939231c1d33cfe6259f1287 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a change in pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on a change in pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#discussion_r797123944
##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFileReader.java
##########
@@ -481,4 +447,63 @@ public long moveToPrev() throws IOException {
public void remove() {
throw new UnsupportedOperationException("Remove not supported for HoodieLogFileReader");
}
+
+ private static Path makeQualified(FileSystem fs, Path path) {
+ return path.makeQualified(fs.getUri(), fs.getWorkingDirectory());
+ }
+
+ /**
+ * Fetch the right {@link FSDataInputStream} to be used by wrapping with required input streams.
+ * @param fs instance of {@link FileSystem} in use.
+ * @param bufferSize buffer size to be used.
+ * @return the right {@link FSDataInputStream} as required.
+ */
+ private static FSDataInputStream getFSDataInputStream(FileSystem fs,
Review comment:
This didn't change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1027682215
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
}, {
"hash" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667",
"triggerID" : "d0b17e523f7d9b4316583ebe8eefe72116a64dd7",
"triggerType" : "PUSH"
}, {
"hash" : "29dd8404768aa1811844efc1332d02041971b8e0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676",
"triggerID" : "29dd8404768aa1811844efc1332d02041971b8e0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d0b17e523f7d9b4316583ebe8eefe72116a64dd7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5667)
* 29dd8404768aa1811844efc1332d02041971b8e0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5676)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996282083
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-996987870
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* dc2724adce7261d76617bf9e77bcd3048d3db200 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393)
* e55e28fa924315ebc1b964f03440e3f3cd02732a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012796670
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012686549
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d2bc41ad938fc196af27f03560a91503d3cbc69c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919)
* d7f63eb9541962516366659ec0c2f86157397c4d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012796670
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
}, {
"hash" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222",
"triggerID" : "2a3b74e239bf96351eece45e928eac8e4f96d4ff",
"triggerType" : "PUSH"
}, {
"hash" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225",
"triggerID" : "9fb948535b79336115a8e0fddfa955f7bfbff5f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2a3b74e239bf96351eece45e928eac8e4f96d4ff Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5222)
* 9fb948535b79336115a8e0fddfa955f7bfbff5f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5225)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-1012726017
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
}, {
"hash" : "573af13b233334d2678ab40ca51c7424757aae60",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4677",
"triggerID" : "573af13b233334d2678ab40ca51c7424757aae60",
"triggerType" : "PUSH"
}, {
"hash" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4894",
"triggerID" : "be81e69fc9e51a211520de6e86efb7f7192a39db",
"triggerType" : "PUSH"
}, {
"hash" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4897",
"triggerID" : "463e9f1675c105a65d1eeb26bf05d2ef06c4216c",
"triggerType" : "PUSH"
}, {
"hash" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4916",
"triggerID" : "8b7962db7e7697450939231c1d33cfe6259f1287",
"triggerType" : "PUSH"
}, {
"hash" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4919",
"triggerID" : "d2bc41ad938fc196af27f03560a91503d3cbc69c",
"triggerType" : "PUSH"
}, {
"hash" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214",
"triggerID" : "d7f63eb9541962516366659ec0c2f86157397c4d",
"triggerType" : "PUSH"
}, {
"hash" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218",
"triggerID" : "3cb5ff6636414fa2ca81c9740e073f694a696e5e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* d7f63eb9541962516366659ec0c2f86157397c4d Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5214)
* 3cb5ff6636414fa2ca81c9740e073f694a696e5e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5218)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999365421
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 2f13b35851760cc2e62a702951041cfabb5453de Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4658)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4333: [HUDI-431] Adding support for Parquet in MOR `LogBlock`s
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4333:
URL: https://github.com/apache/hudi/pull/4333#issuecomment-999327412
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4345",
"triggerID" : "7bce55782061189c539ae7acb1e9b87ade7cdf7f",
"triggerType" : "PUSH"
}, {
"hash" : "286aa8b95627eaaa01114567797186263a830774",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "286aa8b95627eaaa01114567797186263a830774",
"triggerType" : "PUSH"
}, {
"hash" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4349",
"triggerID" : "00df74e3e4f6039008cd26b789218b48e6ea0d6a",
"triggerType" : "PUSH"
}, {
"hash" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4363",
"triggerID" : "1a390d804e170c0fd42014211db65fd44a3d054e",
"triggerType" : "PUSH"
}, {
"hash" : "e722499ee75403ab62f646fdabca1a2c59570164",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e722499ee75403ab62f646fdabca1a2c59570164",
"triggerType" : "PUSH"
}, {
"hash" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "de0d4385394dc5d820964cefc872f099cee7a02b",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4392",
"triggerID" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"triggerType" : "PUSH"
}, {
"hash" : "dc2724adce7261d76617bf9e77bcd3048d3db200",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4393",
"triggerID" : "996281613",
"triggerType" : "MANUAL"
}, {
"hash" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4450",
"triggerID" : "e55e28fa924315ebc1b964f03440e3f3cd02732a",
"triggerType" : "PUSH"
}, {
"hash" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4460",
"triggerID" : "67cbb2f4ab421fb7a90e4c5d1061613ed331c837",
"triggerType" : "PUSH"
}, {
"hash" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4469",
"triggerID" : "cecde3b6734576c5f2863ec2b4b90689600cb746",
"triggerType" : "PUSH"
}, {
"hash" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "93f3baa443153657ebe212f1c1b453776dc4cc82",
"triggerType" : "PUSH"
}, {
"hash" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611",
"triggerID" : "9a6ce246b27893d31fe7503b6782479741fc74ba",
"triggerType" : "PUSH"
}, {
"hash" : "2f13b35851760cc2e62a702951041cfabb5453de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2f13b35851760cc2e62a702951041cfabb5453de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 286aa8b95627eaaa01114567797186263a830774 UNKNOWN
* e722499ee75403ab62f646fdabca1a2c59570164 UNKNOWN
* de0d4385394dc5d820964cefc872f099cee7a02b UNKNOWN
* 93f3baa443153657ebe212f1c1b453776dc4cc82 UNKNOWN
* 9a6ce246b27893d31fe7503b6782479741fc74ba Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4611)
* 2f13b35851760cc2e62a702951041cfabb5453de UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org