You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "1032851561 (via GitHub)" <gi...@apache.org> on 2023/03/10 08:36:58 UTC
[GitHub] [hudi] 1032851561 opened a new pull request, #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
1032851561 opened a new pull request, #8154:
URL: https://github.com/apache/hudi/pull/8154
### Change Logs
The computed column(eg : process_time as proctime() ) will cause a select exception because it is not included in the data file schema, so we need to ignore the computed column
### Problem
```
create table if not exists ods_test_hudi (
`id` int,
`name` string,
`create_date` timestamp(3),
`process_time` AS PROCTIME(),
PRIMARY KEY(`id`) NOT ENFORCED
)
WITH (
'connector'='hudi',
'table.type'='MERGE_ON_READ'
)
env.sqlQuery("select * from ods_test_hudi /*+ OPTIONS('read.start-commit'='earliest')*/").execute().print();
```
exception:
```
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.hudi.table.format.cow.ParquetSplitReaderUtil.lambda$genPartColumnarRowReader$0(ParquetSplitReaderUtil.java:118)
at java.util.stream.IntPipeline$4$1.accept(IntPipeline.java:250)
at java.util.Spliterators$IntArraySpliterator.forEachRemaining(Spliterators.java:1032)
at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.hudi.table.format.cow.ParquetSplitReaderUtil.genPartColumnarRowReader(ParquetSplitReaderUtil.java:120)
```
### Risk level (write none, low medium or high below)
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8154:
URL: https://github.com/apache/hudi/pull/8154#issuecomment-1463580844
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15661",
"triggerID" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6aa5e494711eeced312b35b2042a0a21bf1a877e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15661)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8154:
URL: https://github.com/apache/hudi/pull/8154#issuecomment-1463566835
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6aa5e494711eeced312b35b2042a0a21bf1a877e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8154:
URL: https://github.com/apache/hudi/pull/8154#issuecomment-1464031652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15661",
"triggerID" : "6aa5e494711eeced312b35b2042a0a21bf1a877e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6aa5e494711eeced312b35b2042a0a21bf1a877e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15661)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #8154:
URL: https://github.com/apache/hudi/pull/8154#issuecomment-1464802437
The failed test case should not be caused by this patch: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=15661&view=logs&j=3b6e910d-b98f-5de6-b9cb-1e5ff571f5de&t=30b5aae4-0ea0-5566-42d0-febf71a7061a&l=321013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #8154: [HUDI-5913] Table can not read correctly when computed column is in the midst
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #8154:
URL: https://github.com/apache/hudi/pull/8154
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org