You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/18 08:34:21 UTC
[GitHub] [hudi] xicm opened a new pull request, #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
xicm opened a new pull request, #6986:
URL: https://github.com/apache/hudi/pull/6986
### Change Logs
If hoodie.datasource.write.drop.partition.columns=true, the log reader will not add partition values to record. The query with partition in where clause will return null.
This pr extract partition value from file path and add it to the record.
### Impact
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java
### Risk level (write none, low medium or high below)
low
_If medium or high, explain what verification was done to mitigate the risks._
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
YannByron commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1001864129
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/cdc/TestCDCDataFrameSuite.scala:
##########
@@ -593,8 +593,8 @@ class TestCDCDataFrameSuite extends HoodieCDCTestBase {
// it will write out >1 cdc log files due to rollover.
assert(cdcLogFiles2.size > 1)
// with a small value for 'hoodie.logfile.data.block.max.size',
- // it will write out >1 cdc data blocks in one single cdc log file.
- assert(getCDCBlocks(cdcLogFiles2.head, cdcSchema).size > 1)
+ // it will write out >= 1 cdc data blocks in one single cdc log file.
+ assert(getCDCBlocks(cdcLogFiles2.head, cdcSchema).size >= 1)
Review Comment:
this UT has been fixed, please rebase master branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1290239245
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab5cfec6e37a61ebb2899e4d683ce36e5035cf35 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428)
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286965436
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407)
* ab5cfec6e37a61ebb2899e4d683ce36e5035cf35 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286258240
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284156906
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6c3f427967e17da7ed960a8dc7944f2f3d39fe76 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329)
* 970ad798c5690ef0ddcd8d32e480f49959e16de9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284765831
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344)
* df2c48f1665cbdd49404ce6cfb58c880e2598019 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286972525
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407)
* ab5cfec6e37a61ebb2899e4d683ce36e5035cf35 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [WIP][HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299448812
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286779839
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1291553277
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563)
* 240b1a43e85fb85791e5c9bcb87aa9e95201c1cb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585)
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1290247083
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab5cfec6e37a61ebb2899e4d683ce36e5035cf35 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428)
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
switch (newSchema.getType()) {
case NULL:
case BOOLEAN:
+ if (oldSchema.getType() == Schema.Type.STRING) {
Review Comment:
Hi @alexeykudinkin, thanks for your suggestion very much.
What do you think about wrapping logFiles as PartitionedFile, just like basefiles?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1292072965
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299481483
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711",
"triggerID" : "1299469454",
"triggerType" : "MANUAL"
}, {
"hash" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711)
* cd21d9804de19bc035e6bd8f1665bcd88ffa342f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1285079700
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* df2c48f1665cbdd49404ce6cfb58c880e2598019 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1285181542
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* df2c48f1665cbdd49404ce6cfb58c880e2598019 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359)
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1283926084
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6c3f427967e17da7ed960a8dc7944f2f3d39fe76 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1287054164
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ab5cfec6e37a61ebb2899e4d683ce36e5035cf35 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1002132914
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java:
##########
@@ -195,6 +203,24 @@ protected AbstractHoodieLogRecordReader(FileSystem fs, String basePath, List<Str
Pair.of(tableConfig.getRecordKeyFieldProp(), tableConfig.getPartitionFieldProp()));
}
this.partitionName = partitionName;
+ this.partitionValues = getPartitionValues();
Review Comment:
It's not a good practice to call non-static methods from the ctor, since it relies implicitly on the ordering of the initialization. Let's instead make this method static and pass it all values it needs.
##########
hudi-common/src/test/java/org/apache/hudi/common/functional/TestHoodieLogRecordReader.java:
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.functional;
+
+import org.apache.hudi.avro.HoodieAvroUtils;
+import org.apache.hudi.common.fs.FSUtils;
+import org.apache.hudi.common.model.HoodieAvroPayload;
+import org.apache.hudi.common.model.HoodieLogFile;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.model.HoodieTableType;
+import org.apache.hudi.common.table.HoodieTableConfig;
+import org.apache.hudi.common.table.log.HoodieLogFormat;
+import org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner;
+import org.apache.hudi.common.table.log.block.HoodieAvroDataBlock;
+import org.apache.hudi.common.table.log.block.HoodieDataBlock;
+import org.apache.hudi.common.table.log.block.HoodieLogBlock;
+import org.apache.hudi.common.testutils.FileCreateUtils;
+import org.apache.hudi.common.testutils.HoodieTestUtils;
+import org.apache.hudi.common.testutils.SchemaTestUtil;
+
+import org.apache.avro.Schema;
+import org.apache.avro.generic.GenericRecord;
+import org.apache.avro.generic.IndexedRecord;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.FileUtil;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.DFSConfigKeys;
+import org.apache.hadoop.hdfs.MiniDFSCluster;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.File;
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.UUID;
+import java.util.stream.Collectors;
+
+import static org.apache.hudi.common.testutils.SchemaTestUtil.getSimpleSchema;
+import static org.apache.hudi.common.util.collection.ExternalSpillableMap.DiskMapType.BITCASK;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+public class TestHoodieLogRecordReader {
+
+ private static File dfsBaseDir;
+ private static MiniDFSCluster cluster;
+
+ private FileSystem fs;
+ private String basePath;
+ private Path partitionPath;
+ private String partitionValue = "my_partition";
+ private String partitionName = "partition";
+
+ @BeforeAll
+ public static void setUpClass() throws IOException, InterruptedException {
+ dfsBaseDir = new File("/tmp/" + UUID.randomUUID().toString());
+ FileUtil.fullyDelete(dfsBaseDir);
+ // Append is not supported in LocalFileSystem. HDFS needs to be setup.
+ Configuration conf = new Configuration();
+ // lower heartbeat interval for fast recognition of DN
+ conf.set(MiniDFSCluster.HDFS_MINIDFS_BASEDIR, dfsBaseDir.getAbsolutePath());
Review Comment:
I don't think we really need HDFS for this test. What's the idea behind doing it on HDFS?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java:
##########
@@ -195,6 +203,24 @@ protected AbstractHoodieLogRecordReader(FileSystem fs, String basePath, List<Str
Pair.of(tableConfig.getRecordKeyFieldProp(), tableConfig.getPartitionFieldProp()));
}
this.partitionName = partitionName;
+ this.partitionValues = getPartitionValues();
+ }
+
+ private Option<String[]> getPartitionValues() {
+ if (!this.partitionName.isPresent()) {
+ return Option.empty();
+ }
+
+ String[] partitionValues = this.partitionName.get().split("/");
+ if (this.hiveStylePartition) {
+ return Option.of(Arrays.stream(partitionValues)
+ .map(partition -> partition.split("="))
+ .filter(partition -> partition.length == 2)
Review Comment:
Why do we filter on specific length here?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java:
##########
@@ -632,6 +658,19 @@ private void processDataBlock(HoodieDataBlock dataBlock, Option<KeySpec> keySpec
while (recordIterator.hasNext()) {
IndexedRecord currentRecord = recordIterator.next();
IndexedRecord record = schemaOption.isPresent() ? HoodieAvroUtils.rewriteRecordWithNewSchema(currentRecord, schemaOption.get(), Collections.emptyMap()) : currentRecord;
+
+ Schema schema = record.getSchema();
+ if (this.dropPartitions && this.partitionFields.isPresent() && this.partitionValues.isPresent()) {
+ String[] partitionFields = this.partitionFields.get();
+ String[] partitionValues = this.partitionValues.get();
+
+ if (partitionFields.length == partitionValues.length) {
Review Comment:
This condition should be asserted on during init
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordReader.java:
##########
@@ -632,6 +658,19 @@ private void processDataBlock(HoodieDataBlock dataBlock, Option<KeySpec> keySpec
while (recordIterator.hasNext()) {
IndexedRecord currentRecord = recordIterator.next();
IndexedRecord record = schemaOption.isPresent() ? HoodieAvroUtils.rewriteRecordWithNewSchema(currentRecord, schemaOption.get(), Collections.emptyMap()) : currentRecord;
+
+ Schema schema = record.getSchema();
+ if (this.dropPartitions && this.partitionFields.isPresent() && this.partitionValues.isPresent()) {
+ String[] partitionFields = this.partitionFields.get();
+ String[] partitionValues = this.partitionValues.get();
+
+ if (partitionFields.length == partitionValues.length) {
+ for (int i = 0; i < partitionValues.length; i++) {
+ record.put(schema.getField(partitionFields[i]).pos(), partitionValues[i]);
Review Comment:
We can't just blindly inject strings into record, we need to make sure we coercing it to a type of the column
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1291600094
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 240b1a43e85fb85791e5c9bcb87aa9e95201c1cb Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585)
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299485540
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711",
"triggerID" : "1299469454",
"triggerType" : "MANUAL"
}, {
"hash" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12713",
"triggerID" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711)
* cd21d9804de19bc035e6bd8f1665bcd88ffa342f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12713)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1020536394
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java:
##########
@@ -107,6 +107,33 @@ protected HoodieMergedLogRecordScanner(FileSystem fs, String basePath, List<Stri
}
}
+ @SuppressWarnings("unchecked")
+ protected HoodieMergedLogRecordScanner(FileSystem fs, String basePath, List<String> logFilePaths, Schema readerSchema,
Review Comment:
Let's avoid copying the whole ctor. Instead, generalize existing one and redirect this one to it.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java:
##########
@@ -233,6 +260,8 @@ public static class Builder extends AbstractHoodieLogRecordReader.Builder {
// Use scanV2 method.
private boolean useScanV2 = false;
+ private Object[] partitionValues;
Review Comment:
Let's create an object akin to `PartitionPath` we have in Spark
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/LogFileIterator.scala:
##########
@@ -300,6 +296,10 @@ object LogFileIterator {
if (logFiles.nonEmpty) {
logRecordScannerBuilder.withPartition(
getRelativePartitionPath(new Path(tableState.tablePath), logFiles.head.getPath.getParent))
+
+ if (logFiles.head.isInstanceOf[HoodieLogFileWithPartition]) {
Review Comment:
Please check my comment below
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieLogFileWithPartition.java:
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.model;
+
+public class HoodieLogFileWithPartition extends HoodieLogFile {
Review Comment:
Please check my comment below regarding this class
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala:
##########
@@ -213,7 +214,24 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
PartitionedFile(getPartitionColumnsAsInternalRow(file.getFileStatus), filePath, 0, file.getFileLen)
}
- HoodieMergeOnReadFileSplit(partitionedBaseFile, logFiles)
+ if (shouldExtractPartitionValuesFromPartitionPath && logFiles.nonEmpty) {
Review Comment:
I don't think we actually need this -- log-files always co-located w/ base-file and as such we can extract partition-values from there
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala:
##########
@@ -213,7 +214,24 @@ class MergeOnReadSnapshotRelation(sqlContext: SQLContext,
PartitionedFile(getPartitionColumnsAsInternalRow(file.getFileStatus), filePath, 0, file.getFileLen)
Review Comment:
Let's also update `getPartitionColumnsAsInternalRow` to reuse `parsePartitionColumnValues` instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [WIP][HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299469078
Hi @alexeykudinkin,
I create a new model ```HoodieLogFileWithPartition``` extends ```HoodieLogFile``` (is it a good way?), and init partitionValues when ```buildSplits``` in ```MergeOnReadSnapshotRelation```, finally pass partitionValues to ```AbstractHoodieLogRecordReader```.
Please review when you are available. If there are unreasonable implementations, correct me, I will fix. Thanks. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1290279092
@alexeykudinkin Thanks for your advice. I updated the pr, review again when you are free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1282095935
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b09437530fce0720b9fc35a7f27a8f45cbb2eab UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284165890
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6c3f427967e17da7ed960a8dc7944f2f3d39fe76 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329)
* 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284772334
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344)
* df2c48f1665cbdd49404ce6cfb58c880e2598019 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1282107060
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b09437530fce0720b9fc35a7f27a8f45cbb2eab Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1282491150
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b09437530fce0720b9fc35a7f27a8f45cbb2eab Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299469454
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299477234
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711",
"triggerID" : "1299469454",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 1d34a26c34ced9fead6af9725e68d8bb86cc934e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1299579002
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12589",
"triggerID" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"triggerType" : "PUSH"
}, {
"hash" : "1d34a26c34ced9fead6af9725e68d8bb86cc934e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12711",
"triggerID" : "1299469454",
"triggerType" : "MANUAL"
}, {
"hash" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12713",
"triggerID" : "cd21d9804de19bc035e6bd8f1665bcd88ffa342f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd21d9804de19bc035e6bd8f1665bcd88ffa342f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12713)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1291442579
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563)
* 240b1a43e85fb85791e5c9bcb87aa9e95201c1cb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1007294815
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
switch (newSchema.getType()) {
case NULL:
case BOOLEAN:
+ if (oldSchema.getType() == Schema.Type.STRING) {
Review Comment:
Also i'd suggest you to check out `Spark3ParsePartitionUtil`
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
switch (newSchema.getType()) {
case NULL:
case BOOLEAN:
+ if (oldSchema.getType() == Schema.Type.STRING) {
Review Comment:
Appreciate your approach of extending existing functionality, but i think in that case we actually shouldn't be doing that -- conversions we're doing here aren't "canonical" (there's no permitted conversion from string to int, for ex in Spark) and as such, we'd rather keep this conversion limited in scope to be applicable only to partition values where it's both inevitable and is actually sensible (since we know that we're not trying to convert some random string to int, but the int that was previously converted to a string)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
YannByron commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1282058186
@xicm better to have some UT to know the effect of this changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1283635287
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b09437530fce0720b9fc35a7f27a8f45cbb2eab Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292)
* 6c3f427967e17da7ed960a8dc7944f2f3d39fe76 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
switch (newSchema.getType()) {
case NULL:
case BOOLEAN:
+ if (oldSchema.getType() == Schema.Type.STRING) {
Review Comment:
Hi @alexeykudinkin, thanks for your suggestion very much.
What do you think about wrapping logFiles as PartitionedFile, just like basefiles?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1007788563
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -874,21 +887,30 @@ private static Object rewritePrimaryTypeWithDiffSchemaType(Object oldValue, Sche
switch (newSchema.getType()) {
case NULL:
case BOOLEAN:
+ if (oldSchema.getType() == Schema.Type.STRING) {
Review Comment:
Hi @alexeykudinkin, thanks for your suggestion very much.
What do you think about wrapping logFiles as PartitionedFiles, just like basefiles?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286354627
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1291445930
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
}, {
"hash" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585",
"triggerID" : "240b1a43e85fb85791e5c9bcb87aa9e95201c1cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563)
* 240b1a43e85fb85791e5c9bcb87aa9e95201c1cb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12585)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1291193104
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
}, {
"hash" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12428",
"triggerID" : "ab5cfec6e37a61ebb2899e4d683ce36e5035cf35",
"triggerType" : "PUSH"
}, {
"hash" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563",
"triggerID" : "74787e65d82a88ece5a1a9ece86a207d9ca2125e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74787e65d82a88ece5a1a9ece86a207d9ca2125e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12563)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1341863937
The problem of log reader is caused by setting "hoodie.datasource.write.drop.partition.columns=true", and the purpose of setting "hoodie.datasource.write.drop.partition.columns=true" is to solve the problem of hive query. Hive3 query returns null when the where clause has a partition field. Setting "hoodie.datasource.write.drop.partition.columns=true" can solve this problem.
I created a new pr #7355 to solve the hive query problem. I think we can close this pr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1283561515
> @xicm better to have some UT to know the effect of this changes.
Hi @YannByron I added an UT, have a review if you are free.:)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1284595815
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 970ad798c5690ef0ddcd8d32e480f49959e16de9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1285191079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* df2c48f1665cbdd49404ce6cfb58c880e2598019 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359)
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1286357763
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12329",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
}, {
"hash" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12344",
"triggerID" : "970ad798c5690ef0ddcd8d32e480f49959e16de9",
"triggerType" : "PUSH"
}, {
"hash" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12359",
"triggerID" : "df2c48f1665cbdd49404ce6cfb58c880e2598019",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383",
"triggerID" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"triggerType" : "PUSH"
}, {
"hash" : "088cfee9a091af9f0327d4b432fcb3aa9a6e22ca",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407",
"triggerID" : "1286354627",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 088cfee9a091af9f0327d4b432fcb3aa9a6e22ca Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12383) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12407)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on a diff in pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
xicm commented on code in PR #6986:
URL: https://github.com/apache/hudi/pull/6986#discussion_r1000314284
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/cdc/TestCDCDataFrameSuite.scala:
##########
@@ -593,8 +593,8 @@ class TestCDCDataFrameSuite extends HoodieCDCTestBase {
// it will write out >1 cdc log files due to rollover.
assert(cdcLogFiles2.size > 1)
// with a small value for 'hoodie.logfile.data.block.max.size',
- // it will write out >1 cdc data blocks in one single cdc log file.
- assert(getCDCBlocks(cdcLogFiles2.head, cdcSchema).size > 1)
+ // it will write out >= 1 cdc data blocks in one single cdc log file.
+ assert(getCDCBlocks(cdcLogFiles2.head, cdcSchema).size >= 1)
Review Comment:
Flaky test, the blocks may be 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6986:
URL: https://github.com/apache/hudi/pull/6986#issuecomment-1283626635
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292",
"triggerID" : "8b09437530fce0720b9fc35a7f27a8f45cbb2eab",
"triggerType" : "PUSH"
}, {
"hash" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6c3f427967e17da7ed960a8dc7944f2f3d39fe76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b09437530fce0720b9fc35a7f27a8f45cbb2eab Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12292)
* 6c3f427967e17da7ed960a8dc7944f2f3d39fe76 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin closed pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
Posted by GitBox <gi...@apache.org>.
alexeykudinkin closed pull request #6986: [HUDI-5047] Add partition value in HoodieLogRecordReader when hoodie.datasource.write.drop.partition.columns=true
URL: https://github.com/apache/hudi/pull/6986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org