You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "danny0405 (via GitHub)" <gi...@apache.org> on 2023/02/07 06:48:01 UTC
[GitHub] [hudi] danny0405 opened a new pull request, #7874: [HUDI-5718] Unsupported Operation Exception for compaction
danny0405 opened a new pull request, #7874:
URL: https://github.com/apache/hudi/pull/7874
### Change Logs
In HoodieMergeHandle, there is such a code snippet:
```java
protected void writeIncomingRecords() throws IOException {
// write out any pending records (this can happen when inserts are turned into updates)
Iterator<HoodieRecord<T>> newRecordsItr = (keyToNewRecords instanceof ExternalSpillableMap)
? ((ExternalSpillableMap)keyToNewRecords).iterator() : keyToNewRecords.values().iterator();
while (newRecordsItr.hasNext()) {
HoodieRecord<T> hoodieRecord = newRecordsItr.next();
if (!writtenRecordKeys.contains(hoodieRecord.getRecordKey())) {
writeInsertRecord(hoodieRecord);
}
}
}
```
, after https://github.com/apache/hudi/commit/f8028a400ebe7a64d7745fae90e769306a8a07a7, the map is changes to be as immutabe, thus `keyToNewRecords.values()` is invoked and throws error!
### Impact
FIx the compaction code path
### Risk level (write none, low medium or high below)
low
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7874:
URL: https://github.com/apache/hudi/pull/7874#issuecomment-1420296365
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4cb403fe910bd6d840844d1fea436bc83158359 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7874:
URL: https://github.com/apache/hudi/pull/7874#issuecomment-1420948665
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984",
"triggerID" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"triggerType" : "PUSH"
}, {
"hash" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14988",
"triggerID" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ed593981a202e1534c6bd52b6e316ddcb57d5c84 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14988)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7874:
URL: https://github.com/apache/hudi/pull/7874#issuecomment-1420524055
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984",
"triggerID" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"triggerType" : "PUSH"
}, {
"hash" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14988",
"triggerID" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4cb403fe910bd6d840844d1fea436bc83158359 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984)
* ed593981a202e1534c6bd52b6e316ddcb57d5c84 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14988)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7874:
URL: https://github.com/apache/hudi/pull/7874#issuecomment-1420302800
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984",
"triggerID" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4cb403fe910bd6d840844d1fea436bc83158359 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7874:
URL: https://github.com/apache/hudi/pull/7874#issuecomment-1420511457
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984",
"triggerID" : "f4cb403fe910bd6d840844d1fea436bc83158359",
"triggerType" : "PUSH"
}, {
"hash" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ed593981a202e1534c6bd52b6e316ddcb57d5c84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f4cb403fe910bd6d840844d1fea436bc83158359 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14984)
* ed593981a202e1534c6bd52b6e316ddcb57d5c84 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #7874: [HUDI-5718] Unsupported Operation Exception for compaction
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope merged PR #7874:
URL: https://github.com/apache/hudi/pull/7874
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org