You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "beryllw (via GitHub)" <gi...@apache.org> on 2026/08/27 07:47:29 UTC
[I] Lake tiering fails for append-only tables with upper-case column names [fluss]
beryllw opened a new issue, #4122:
URL: https://github.com/apache/fluss/issues/4122
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
main (development)
### Please describe the bug 🐞
Tiering an append-only (no primary key) table to Paimon fails when any column name contains an
upper-case letter.
Reproduce:
```sql
CREATE TABLE test_log (
`event_key` STRING NOT NULL,
`eventId` STRING
) WITH (
'bucket.num' = '1',
'bucket.key' = 'event_key',
'table.datalake.enabled' = 'true',
'table.datalake.format' = 'paimon'
);
INSERT INTO test_log VALUES ('k1', 'e1');
```
The tiering job then fails on every batch:
```
java.io.IOException: Failed to write Arrow record batch to Paimon.
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: field eventid not found in
[event_key: Utf8, eventId: Utf8, __bucket: ..., __offset: ..., __timestamp: ...]
at org.apache.paimon.arrow.reader.ArrowBatchReader.readBatch(ArrowBatchReader.java:69)
at org.apache.fluss.lake.paimon.tiering.append.AppendOnlyArrowBatchHelper.writeArrowBatch(AppendOnlyArrowBatchHelper.java:133)
```
Cause: AppendOnlyArrowBatchHelper maps Paimon columns onto the Arrow vectors
case-insensitively, while the Arrow schema keeps the original Fluss column names.Primary key tables are not affected, they map columns by position.
### Solution
Map the columns case-sensitively in `AppendOnlyArrowBatchHelper`, and cover the Arrow record
batch write path with tests for upper-case column names.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [lake/tiering]Lake tiering fails for append-only tables with upper-case column names [fluss]
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia closed issue #4122: [lake/tiering]Lake tiering fails for append-only tables with upper-case column names
URL: https://github.com/apache/fluss/issues/4122
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org