You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2026/01/19 16:50:07 UTC
[PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Zouxxyy opened a new pull request, #7081:
URL: https://github.com/apache/paimon/pull/7081
<!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #xxx
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #7081:
URL: https://github.com/apache/paimon/pull/7081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on PR #7081:
URL: https://github.com/apache/paimon/pull/7081#issuecomment-3773273111
Important Note:
Parquet 1.16.0 has modified `HadoopInputFile` in `parquet-hadoop`, which now requires `FutureDataInputStreamBuilder` from hadoop-common in Hadoop 3.x. However, Paimon is compiled by default against Hadoop version 2.8.
```
2026-01-20T03:47:05.9386467Z Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FileSystem.openFile(Lorg/apache/hadoop/fs/Path;)Lorg/apache/hadoop/fs/FutureDataInputStreamBuilder;
2026-01-20T03:47:05.9386725Z at org.apache.parquet.hadoop.util.HadoopInputFile.newStream(HadoopInputFile.java:114)
2026-01-20T03:47:05.9386977Z at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:563)
2026-01-20T03:47:05.9387218Z at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:508)
2026-01-20T03:47:05.9387458Z at org.apache.parquet.hadoop.ParquetFileReader.readFooter(ParquetFileReader.java:490)
2026-01-20T03:47:05.9387794Z at org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:79)
2026-01-20T03:47:05.9388159Z at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:75)
2026-01-20T03:47:05.9388517Z at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:60)
2026-01-20T03:47:05.9388889Z at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:75)
2026-01-20T03:47:05.9389377Z at org.apache.flink.connectors.hive.HiveTableMetaStoreFactory$HiveTableMetaStore.gatherFullStats(HiveTableMetaStoreFactory.java:272)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy closed pull request #7081: [parquet] Bump parquet version to 1.16.0
URL: https://github.com/apache/paimon/pull/7081
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Posted by "Zouxxyy (via GitHub)" <gi...@apache.org>.
Zouxxyy commented on PR #7081:
URL: https://github.com/apache/paimon/pull/7081#issuecomment-3783392687
Port this patch
```
ParquetFileReader
GH-3249: Fix incorrect Bloom filter data when reading from ByteArrayInputStream by using readFully()
GH-3207: ParquetFileReader supports detachFileInputStream
MINOR: Reader fails fast when footer size is larger than INT_MAX
GH-3141: Add constructor to `ParquetFileReader` to pass parquet footer and expose setRequestedSchema that accepts `List<ColumnDescriptor>`
ParquetWriter
GH-3235: Row count limit for each row group
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [parquet] Bump parquet version to 1.16.0 [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #7081:
URL: https://github.com/apache/paimon/pull/7081#issuecomment-3776684161
We should also update copied classes from parquet new version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org