You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Ayush Saxena (Jira)" <ji...@apache.org> on 2024/01/30 22:19:00 UTC
[jira] [Commented] (HIVE-28048) Hive cannot run ORDER BY queries on Iceberg tables partitioned by decimal columns
[ https://issues.apache.org/jira/browse/HIVE-28048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17812496#comment-17812496 ]
Ayush Saxena commented on HIVE-28048:
-------------------------------------
if that is only decimal, maybe we can do a convert for bigdecimal and handle this, that is what comes into my mind and what even works
{noformat}
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java
index 0abbd59b9e..7b202c4977 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java
@@ -18,6 +18,7 @@
package org.apache.hadoop.hive.ql.exec.vector;
import java.io.IOException;
+import java.math.BigDecimal;
import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Map;
@@ -36,6 +37,7 @@
import org.apache.hadoop.hive.ql.io.BucketIdentifier;
import org.apache.hadoop.hive.ql.io.HiveFileFormatUtils;
import org.apache.hadoop.hive.ql.io.IOPrepareCache;
+import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.metadata.VirtualColumn;
import org.apache.hadoop.hive.ql.plan.MapWork;
@@ -585,7 +587,11 @@ public void addPartitionColsToBatch(ColumnVector col, Object value, int colIndex
dv.isNull[0] = true;
dv.isRepeating = true;
} else {
- dv.fill((HiveDecimal) value);
+ if (value instanceof BigDecimal) {
+ dv.fill(HiveDecimal.create((BigDecimal) value));
+ } else {
+ dv.fill((HiveDecimal) value);
+ }
}
}
}
{noformat}
but if it is other than decimal, maybe for partition value, we should do that getPrimitiveJavaObject logic by extracting some stuff from {{IcebergObjectInspector}} {{primitive}} method
> Hive cannot run ORDER BY queries on Iceberg tables partitioned by decimal columns
> ---------------------------------------------------------------------------------
>
> Key: HIVE-28048
> URL: https://issues.apache.org/jira/browse/HIVE-28048
> Project: Hive
> Issue Type: Bug
> Reporter: Zoltán Borók-Nagy
> Priority: Major
> Labels: iceberg
>
> Repro:
> {noformat}
> create table test_dec (d decimal(8,4), i int)
> partitioned by spec (d)
> stored by iceberg;
> insert into test_dec values (3.4, 5), (4.5, 6);
> select * from test_dec order by i;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)