You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/09 14:14:00 UTC
[jira] [Commented] (IMPALA-11854) ImpalaStringWritable's underlying array can't be changed in UDFs
[ https://issues.apache.org/jira/browse/IMPALA-11854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698404#comment-17698404 ]
ASF subversion and git services commented on IMPALA-11854:
----------------------------------------------------------
Commit afe59f7f0d5658a6639e3e6f1f9ff9ed2f76d658 in impala's branch refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=afe59f7f0 ]
IMPALA-11854: ImpalaStringWritable's underlying array can't be changed in UDFs
This change fixes the behavior of BytesWritable and TextWritable's
getBytes() method. Now the returned byte array could be handled as
the underlying buffer as it gets loaded before the UDF's evaluation,
and tracks the changes as a regular Java byte array; the resizing
operation still resets the reference. The operations that wrote back
to the native heap were also removed as these operations are now
handled in the byte array. ImpalaStringWritable class is also removed,
writables that used it before now store the data directly.
Tests:
- Test UDFs added as BufferAlteringUdf and GenericBufferAlteringUdf
- E2E test ran for UDFs
Change-Id: Ifb28bd0dce7b0482c7abe1f61f245691fcbfe212
Reviewed-on: http://gerrit.cloudera.org:8080/19507
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> ImpalaStringWritable's underlying array can't be changed in UDFs
> ----------------------------------------------------------------
>
> Key: IMPALA-11854
> URL: https://issues.apache.org/jira/browse/IMPALA-11854
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.3.0
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
>
> ImpalaStringWritable's getBytes() implementation returns the copy of the underlying buffer, therefore the original value can't be changed. This behavior makes UDFs work erroneously when they contain operations that would change the BINARY/TEXT arguments. For example:
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/esri/ST_SetSRID.java#L47]
> where the incoming `geomref` argument should change, but it gets returned unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org