You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Alessandro D'Armiento (JIRA)" <ji...@apache.org> on 2019/08/03 11:42:00 UTC
[jira] [Comment Edited] (NIFI-6513) The property of
PutHive3Streaming Processor named "Auto-Create Partitions" is useless
[ https://issues.apache.org/jira/browse/NIFI-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899443#comment-16899443 ]
Alessandro D'Armiento edited comment on NIFI-6513 at 8/3/19 11:41 AM:
----------------------------------------------------------------------
Are we sure we actually need that boolean?
In case you enable dynamic partitioning, you should be able to set the maximum number of partitions in the hive configuration files you are able to pass to the `Hive Configuration Resources` property. Furthermore, HiveConnection[1] isDynamic() method only checks for the existence of a list of static partitions (as well as if the table itself is partitioned).
If you instead filled the `Partition Values` you are telling Hive you are using static partitioning and no partitions should be created anyhow.
I suggest renaming `Partition Values` with `Static Partition Values` for correctness and remove that boolean
cc [~mattyb149], which worked on the Hive Processor in NIFI-4963
[1] https://github.com/apache/hive/blob/d7475aa98f6a2fc813e2e1c0ad99f902cb28cc00/streaming/src/java/org/apache/hive/streaming/HiveStreamingConnection.java#L867
was (Author: axelsync):
Are we sure we actually need that boolean?
In case you enable dynamic partitioning, you should be able to set the maximum number of partitions in the hive configuration files you are able to pass to the `Hive Configuration Resources` property. If you instead filled the `Partition Values` you are telling Hive you are using static partitioning and no partitions should be created anyhow.
I suggest renaming `Partition Values` with `Static Partition Values` for correctness and remove that boolean
cc [~mattyb149], which worked on the Hive Processor in NIFI-4963
> The property of PutHive3Streaming Processor named "Auto-Create Partitions" is useless
> -------------------------------------------------------------------------------------
>
> Key: NIFI-6513
> URL: https://issues.apache.org/jira/browse/NIFI-6513
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.9.0
> Environment: Hive3.1.0
> Reporter: Lu Wang
> Priority: Major
>
> _emphasized text_The PutHive3Streaming processor always creates the partition, regardless of whether the value of property named "Auto-Create Partitions" is set to true or false。
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)