You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/12 13:19:00 UTC
[jira] [Work logged] (HIVE-25686) UDFSpace result length calculation is incorrect after HADOOP-17901 and HADOOP-17905
[ https://issues.apache.org/jira/browse/HIVE-25686?focusedWorklogId=680810&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-680810 ]
ASF GitHub Bot logged work on HIVE-25686:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Nov/21 13:18
Start Date: 12/Nov/21 13:18
Worklog Time Spent: 10m
Work Description: mbathori-cloudera opened a new pull request #2786:
URL: https://github.com/apache/hive/pull/2786
### What changes were proposed in this pull request?
After the application of HADOOP-17901 and HADOOP-17905 they changed the backing array increment logic which is causing issues in the UDFSpace class.
### Why are the changes needed?
The previous `result.getBytes().length` is returning incorrect size which is causing error in the result output. By changing this method call to `result.getLength()` the size calculation is correct again any the original behaviour is restored.
Example of the UDFSpace behaviour with the different method calls:
```
space parameter - 11
result.getBytes().length - 0
result.getLength() - 0
space parameter - 12
result.getBytes().length - 11
result.getLength() - 11
space parameter - 13
result.getBytes().length - 16
result.getLength(): 12
space parameter - 14
result.getBytes().length - 16
result.getLength() - 13
```
Note that the issue is only present whit Hadoop version that contains HADOOP-17901 and HADOOP-17905.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
The change was tested on live cluster and also in the new qtest that was introduced in the scope of this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 680810)
Remaining Estimate: 0h
Time Spent: 10m
> UDFSpace result length calculation is incorrect after HADOOP-17901 and HADOOP-17905
> -----------------------------------------------------------------------------------
>
> Key: HIVE-25686
> URL: https://issues.apache.org/jira/browse/HIVE-25686
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Mark Bathori
> Assignee: Mark Bathori
> Priority: Critical
> Fix For: 4.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The changes added in HADOOP-17901 and HADOOP-17905 introduced some issues in Hive side. UDFSpace is generating false output because the text length is not determined correctly. It is causing unexpected characters in the result.
> *Repro steps:*
> {code:java}
> create table t(i int);{code}
> {code:java}
> insert into t values (5),(6),(7),(8),(9),(10),(11),(12),(13),(14);{code}
> {code:java}
> select i, SPACE(i) from t;{code}
>
> Faulty output:
> {code:java}
> +-----+-----------------+
> | i | _c1 |
> +-----+-----------------+
> | 5 | |
> | 6 | |
> | 7 | |
> | 8 | |
> | 9 | |
> | 10 | |
> | 11 | |
> | 12 | |
> | 13 | |
> | 14 | |
> +-----+-----------------+
> {code}
>
> Also by using {color:#ff0000}--outputformat=csv{color} the generated file contains 0 ascii characters(“EOF”) instead of spaces.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)