You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/20 03:47:34 UTC
[GitHub] [hudi] wzx140 opened a new pull request, #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
wzx140 opened a new pull request, #5907:
URL: https://github.com/apache/hudi/pull/5907
fix ByteBuffer forget to rewind after get in AvroDeserializer
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
fix ByteBuffer forget to rewind after get in AvroDeserializer
## Brief change log
add ByteBuffer#rewind after ByteBuffer#get in AvroDeserializer
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166789189
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "1165322053",
"triggerType" : "MANUAL"
}, {
"hash" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
* fd115fdaf9035e0ba6dcc63d267dc9da57383ff9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1163941979
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0738dc06bdabbd3838d819d921a1122729e36c7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385)
* 38a841fd028140021c4f3d4e77aedf1ed749a5e8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166780284
@danny0405 Thanks. I have rebased master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166791532
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "1165322053",
"triggerType" : "MANUAL"
}, {
"hash" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9570",
"triggerID" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
* fd115fdaf9035e0ba6dcc63d267dc9da57383ff9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9570)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165205144
@nsivabalan I have opened a [pr](https://github.com/apache/spark/pull/36973) in spark. I will post comment in time with news from spark community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165252504
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on a diff in pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on code in PR #5907:
URL: https://github.com/apache/hudi/pull/5907#discussion_r901368557
##########
hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -146,6 +146,8 @@ class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType) {
case b: ByteBuffer =>
val bytes = new Array[Byte](b.remaining)
b.get(bytes)
+ // Do not forget to reset the position
+ b.rewind()
Review Comment:
not yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1163940471
@xushiyan I add an ut about it. The problem is that the pos in ByteBuffer is not set to zero.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1170699303
@nsivabalan @xushiyan Would you like to review this bug fix again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1163944480
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0738dc06bdabbd3838d819d921a1122729e36c7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385)
* 38a841fd028140021c4f3d4e77aedf1ed749a5e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165179540
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 38a841fd028140021c4f3d4e77aedf1ed749a5e8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464)
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165334468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1165322053",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165338185
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "1165322053",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166984032
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9492",
"triggerID" : "1165322053",
"triggerType" : "MANUAL"
}, {
"hash" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9570",
"triggerID" : "fd115fdaf9035e0ba6dcc63d267dc9da57383ff9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fd115fdaf9035e0ba6dcc63d267dc9da57383ff9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9570)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #5907:
URL: https://github.com/apache/hudi/pull/5907#discussion_r903783911
##########
hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -146,6 +146,8 @@ class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType) {
case b: ByteBuffer =>
val bytes = new Array[Byte](b.remaining)
b.get(bytes)
+ // Do not forget to reset the position
+ b.rewind()
Review Comment:
> // Do not forget to reset the position
this would be a helpful note if you can add more details, or better have a UT cover the problem's case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
danny0405 merged PR #5907:
URL: https://github.com/apache/hudi/pull/5907
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5907:
URL: https://github.com/apache/hudi/pull/5907#discussion_r901292535
##########
hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -146,6 +146,8 @@ class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType) {
case b: ByteBuffer =>
val bytes = new Array[Byte](b.remaining)
b.get(bytes)
+ // Do not forget to reset the position
+ b.rewind()
Review Comment:
The clazz is copied from spark, does spark have the same fix ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166769015
Hi, @wzx140 , you may need to rebase the latest master to make the CI build pass, there are some flaky tests on master that have been fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166969625
CI passed in https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=9570
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1160133753
So what's the problem here ? Is there any possibility that we write a test case here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1164034322
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 38a841fd028140021c4f3d4e77aedf1ed749a5e8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on a diff in pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on code in PR #5907:
URL: https://github.com/apache/hudi/pull/5907#discussion_r903794974
##########
hudi-spark-datasource/hudi-spark2/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -146,6 +146,8 @@ class AvroDeserializer(rootAvroType: Schema, rootCatalystType: DataType) {
case b: ByteBuffer =>
val bytes = new Array[Byte](b.remaining)
b.get(bytes)
+ // Do not forget to reset the position
+ b.rewind()
Review Comment:
I will add an ut about it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1163148928
> So what's the problem here ? Is there any possibility that we write a test case here ?
Sorry for so late to reply. I will add an ut about it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1159937820
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0738dc06bdabbd3838d819d921a1122729e36c7f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1159961512
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0738dc06bdabbd3838d819d921a1122729e36c7f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1159993884
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0738dc06bdabbd3838d819d921a1122729e36c7f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165322053
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1165177412
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9385",
"triggerID" : "0738dc06bdabbd3838d819d921a1122729e36c7f",
"triggerType" : "PUSH"
}, {
"hash" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464",
"triggerID" : "38a841fd028140021c4f3d4e77aedf1ed749a5e8",
"triggerType" : "PUSH"
}, {
"hash" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 38a841fd028140021c4f3d4e77aedf1ed749a5e8 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=9464)
* a4f05d0237b98d84e9042c46e3e1de9c1c3cafcb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wzx140 commented on pull request #5907: [HUDI-4285] add ByteBuffer#rewind after ByteBuffer#get in AvroDeseria…
Posted by GitBox <gi...@apache.org>.
wzx140 commented on PR #5907:
URL: https://github.com/apache/hudi/pull/5907#issuecomment-1166747477
@nsivabalan Spark has accepted this change. https://github.com/apache/spark/pull/36973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org