[GitHub] [iceberg] rdblue commented on a change in pull request #2153: Add truncate transform support for MERGE INTO

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #2153:
URL: https://github.com/apache/iceberg/pull/2153#discussion_r564734739



##########
File path: spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/expressions/TransformExpressions.scala
##########
@@ -119,3 +118,38 @@ case class IcebergBucketTransform(numBuckets: Int, child: Expression) extends Ic
 
   override def dataType: DataType = IntegerType
 }
+
+case class IcebergTruncateTransform(child: Expression, width: Int) extends IcebergTransformExpression {
+
+  override def children: Seq[Expression] = child :: Nil
+
+  @transient lazy val truncateFunc: Any => Any = child.dataType match {
+    case _: DecimalType =>
+      val t = Transforms.truncate[Any](icebergInputType, width)
+      d: Any =>  Decimal.apply(t(d.asInstanceOf[Decimal].toJavaBigDecimal).asInstanceOf[java.math.BigDecimal])
+    case _: StringType =>
+      val t = Transforms.truncate[String](Types.StringType.get(), width)
+      s: Any => UTF8String.fromString(t(s.toString))

Review comment:
       `TruncateString` accepts a `CharSequence`, not just `String`. It's unfortunate that `UTF8String` doesn't implement `CharSequence`, but we can fairly easily get one from the underlying bytes: `StandardCharsets.UTF_8.decode(ByteBuffer.wrap(s.asInstanceOf[UTF8String].getBytes))`
   
   That will return a `CharSequence` that is not a copy of the bytes, then we can convert that back to a byte buffer and create the `UTF8String` from that.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org