You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/01/04 09:17:00 UTC
[GitHub] [hudi] manojpec opened a new pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when records by keys
manojpec opened a new pull request #4505:
URL: https://github.com/apache/hudi/pull/4505
## What is the purpose of the pull request
- HoodieMetadataMergedLogRecordReader#getRecordsByKeys() and its parent class methods
are not thread safe. When multiple queries come in for gettting log records
by keys, they all operate on the same log record reader instance provided by
HoodieBackedTableMetadata#openReadersIfNeeded() and they trip over each other
as they clear/put/get the same class memeber records.
## Brief change log
- The fix is to streamline the mutatation to class member records. Making
HoodieMetadataMergedLogRecordReader#getRecordsByKeys() a synchronized method
to avoid concurrent log records readers getting into NPE.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004684486
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004646987
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005100016
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
}, {
"hash" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885",
"triggerID" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0979c3974942226999ab7d6f5f92c07724fa1130 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004644941
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
nsivabalan merged pull request #4505:
URL: https://github.com/apache/hudi/pull/4505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004644941
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005053857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
}, {
"hash" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885",
"triggerID" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
* 0979c3974942226999ab7d6f5f92c07724fa1130 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004646987
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1004684486
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005053857
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
}, {
"hash" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885",
"triggerID" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
* 0979c3974942226999ab7d6f5f92c07724fa1130 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4885)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005051638
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
}, {
"hash" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
* 0979c3974942226999ab7d6f5f92c07724fa1130 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4505: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when reading records by keys
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4505:
URL: https://github.com/apache/hudi/pull/4505#issuecomment-1005051638
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880",
"triggerID" : "15c4b754f2f0a5d085e3b372abbfbba4fa24c69d",
"triggerType" : "PUSH"
}, {
"hash" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0979c3974942226999ab7d6f5f92c07724fa1130",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 15c4b754f2f0a5d085e3b372abbfbba4fa24c69d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=4880)
* 0979c3974942226999ab7d6f5f92c07724fa1130 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org