You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Denys Kuzmenko (Jira)" <ji...@apache.org> on 2026/07/07 09:32:00 UTC
[jira] [Updated] (HIVE-29703) LLAP: Parquet footer larger than alloc.max fails to cache — RuntimeException: Trying to allocate N; max is M
[ https://issues.apache.org/jira/browse/HIVE-29703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Denys Kuzmenko updated HIVE-29703:
----------------------------------
Description:
When LLAP caches a Parquet file footer whose size exceeds hive.llap.io.allocator.alloc.max, the query fails with:
{code}
java.lang.RuntimeException: Trying to allocate 35405479; max is 33554432
at org.apache.hadoop.hive.llap.cache.BuddyAllocator.checkAllocationSize(BuddyAllocator.java:497)
at org.apache.hadoop.hive.llap.cache.BuddyAllocator.allocateMultiple(BuddyAllocator.java:317)
at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.wrapBbForFile(MetadataCache.java:282)
at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.putFileMetadata(MetadataCache.java:241)
at org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl.getParquetFooterBuffersFromCache(LlapIoImpl.java:506)
at org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.readSplitFooter(...)
{code}
was:
When LLAP caches a Parquet file footer whose size exceeds hive.llap.io.allocator.alloc.max, the query fails with:
````
java.lang.RuntimeException: Trying to allocate 35405479; max is 33554432
at org.apache.hadoop.hive.llap.cache.BuddyAllocator.checkAllocationSize(BuddyAllocator.java:497)
at org.apache.hadoop.hive.llap.cache.BuddyAllocator.allocateMultiple(BuddyAllocator.java:317)
at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.wrapBbForFile(MetadataCache.java:282)
at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.putFileMetadata(MetadataCache.java:241)
at org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl.getParquetFooterBuffersFromCache(LlapIoImpl.java:506)
at org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.readSplitFooter(...)
````
> LLAP: Parquet footer larger than alloc.max fails to cache — RuntimeException: Trying to allocate N; max is M
> ------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-29703
> URL: https://issues.apache.org/jira/browse/HIVE-29703
> Project: Hive
> Issue Type: Bug
> Reporter: Denys Kuzmenko
> Priority: Major
>
> When LLAP caches a Parquet file footer whose size exceeds hive.llap.io.allocator.alloc.max, the query fails with:
> {code}
> java.lang.RuntimeException: Trying to allocate 35405479; max is 33554432
> at org.apache.hadoop.hive.llap.cache.BuddyAllocator.checkAllocationSize(BuddyAllocator.java:497)
> at org.apache.hadoop.hive.llap.cache.BuddyAllocator.allocateMultiple(BuddyAllocator.java:317)
> at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.wrapBbForFile(MetadataCache.java:282)
> at org.apache.hadoop.hive.llap.io.metadata.MetadataCache.putFileMetadata(MetadataCache.java:241)
> at org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl.getParquetFooterBuffersFromCache(LlapIoImpl.java:506)
> at org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.readSplitFooter(...)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)