You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2021/02/01 05:39:00 UTC
[jira] [Updated] (HIVE-24710) PTFRowContainer could be reading more
number of blocks than needed
[ https://issues.apache.org/jira/browse/HIVE-24710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajesh Balamohan updated HIVE-24710:
------------------------------------
Description:
PTFRowContainer could be reading the same block repeatedly for the first block. Default block size is around 25000. For the first 25000 rowIdx, it would read the block repeatedly due to ("rowIdx < currentReadBlockStartRow ") condition.
{noformat}
public Row getAt(int rowIdx) throws HiveException {
int blockSize = getBlockSize();
if ( rowIdx < currentReadBlockStartRow || rowIdx >= currentReadBlockStartRow + blockSize ) {
readBlock(getBlockNum(rowIdx));
}
return getReadBlockRow(rowIdx - currentReadBlockStartRow);
}
{noformat}
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/PTFRowContainer.java#L167
was:
PTFRowContainer could be reading the same block repeatedly for the first block. Default block size is around 25000. For the first 25000 rowIdx, it would read the block repeatedly due to ("rowIdx < currentReadBlockStartRow ") condition.
{noformat}
public Row getAt(int rowIdx) throws HiveException {
int blockSize = getBlockSize();
if ( rowIdx < currentReadBlockStartRow || rowIdx >= currentReadBlockStartRow + blockSize ) {
readBlock(getBlockNum(rowIdx));
}
return getReadBlockRow(rowIdx - currentReadBlockStartRow);
}
{noformat}
> PTFRowContainer could be reading more number of blocks than needed
> ------------------------------------------------------------------
>
> Key: HIVE-24710
> URL: https://issues.apache.org/jira/browse/HIVE-24710
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Rajesh Balamohan
> Priority: Major
> Labels: performance
>
> PTFRowContainer could be reading the same block repeatedly for the first block. Default block size is around 25000. For the first 25000 rowIdx, it would read the block repeatedly due to ("rowIdx < currentReadBlockStartRow ") condition.
> {noformat}
> public Row getAt(int rowIdx) throws HiveException {
> int blockSize = getBlockSize();
> if ( rowIdx < currentReadBlockStartRow || rowIdx >= currentReadBlockStartRow + blockSize ) {
> readBlock(getBlockNum(rowIdx));
> }
> return getReadBlockRow(rowIdx - currentReadBlockStartRow);
> }
> {noformat}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/PTFRowContainer.java#L167
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)