You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Shivam (Jira)" <ji...@apache.org> on 2020/02/06 02:22:00 UTC
[jira] [Comment Edited] (HIVE-22771) Partition location incorrectly
formed in FileOutputCommitterContainer
[ https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17031198#comment-17031198 ]
Shivam edited comment on HIVE-22771 at 2/6/20 2:21 AM:
-------------------------------------------------------
I also agree with Daniel's point, that scientific notation is surprising. However, I am unable to understand the benefit we will get by making the id a random String when compared with a random numeric.
Am I missing something?
was (Author: shivam-mohan):
I also agree with Daniel's point, that scientific notation is surprising. However, I am unable to understand the benefit we will get by making the id a random String from a random numeric.
Am I missing something?
> Partition location incorrectly formed in FileOutputCommitterContainer
> ---------------------------------------------------------------------
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
> Affects Versions: 1.2.1
> Reporter: Shivam
> Assignee: Shivam
> Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR + SCRATCH_DIR_NAME + "{color:#ff0000}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)