You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Shivam (Jira)" <ji...@apache.org> on 2020/01/24 09:21:00 UTC
[jira] [Created] (HIVE-22771) Partition location incorrectly formed
in FileOutputCommitterContainer
Shivam created HIVE-22771:
-----------------------------
Summary: Partition location incorrectly formed in FileOutputCommitterContainer
Key: HIVE-22771
URL: https://issues.apache.org/jira/browse/HIVE-22771
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 1.2.1
Reporter: Shivam
Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses function _setOutput_ to generate _idHash_ using below statement:
+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+
*line 116: idHash = String.valueOf(Math.random());*
The output of idHash can be similar to values like this : 7.145347157239135E-4
And,
in class _FileOutputCommitterContainer_ in package _org.apache.hive.hcatalog.mapreduce;_
Uses below statement to compute final partition path:
+*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
*line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR + SCRATCH_DIR_NAME + "{color:#FF0000}\\d\\.?\\d+"{color},"");*
*line 367: partPath = new Path(finalLocn);*
Regex used here is incorrect, since it will only remove integers after the *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)