You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Ilya Kovalev (Jira)" <ji...@apache.org> on 2019/12/24 18:15:00 UTC
[jira] [Updated] (NIFI-6970) Add DistributeRecord processor for
distribute data by key hash
[ https://issues.apache.org/jira/browse/NIFI-6970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ilya Kovalev updated NIFI-6970:
-------------------------------
Description:
Necessary to add processor for {color:#00875a}Record{color} distribution.
Processor must have next fields :
reader, writer, keys (list of {color:#00875a}Record{color} fields for hashing), hash function name.
and an arbitrary number of dynamic properties representing relationships. Also relationships would have weights.
if we have one key and this key is integer then hash function does not evaluate. If we have several keys then cast record values to String and join them with "-" delimiter
like "34-NY-open".
Hash function must return {color:#FF0000}int{color} value. Next processor find {color:#172b4d}+target relationship+ = {color}(hashResult % sum(weights)) and move record according integer range for relationships,
so if we have 2 relationships with weights 4 and 7 appropriately then we have 2 intervals [0, 4) and [4, 11) for first and second relationship (order of relationships matters)
was:
Necessary to add processor for record distribution.
Processor must have next fields :
reader, writer, keys (list of record fields for hashing), hash function name.
and an arbitrary number of dynamic properties representing relationships.
> Add DistributeRecord processor for distribute data by key hash
> --------------------------------------------------------------
>
> Key: NIFI-6970
> URL: https://issues.apache.org/jira/browse/NIFI-6970
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Affects Versions: 1.10.0
> Reporter: Ilya Kovalev
> Priority: Minor
> Fix For: 1.11.0
>
>
> Necessary to add processor for {color:#00875a}Record{color} distribution.
> Processor must have next fields :
> reader, writer, keys (list of {color:#00875a}Record{color} fields for hashing), hash function name.
> and an arbitrary number of dynamic properties representing relationships. Also relationships would have weights.
> if we have one key and this key is integer then hash function does not evaluate. If we have several keys then cast record values to String and join them with "-" delimiter
> like "34-NY-open".
> Hash function must return {color:#FF0000}int{color} value. Next processor find {color:#172b4d}+target relationship+ = {color}(hashResult % sum(weights)) and move record according integer range for relationships,
> so if we have 2 relationships with weights 4 and 7 appropriately then we have 2 intervals [0, 4) and [4, 11) for first and second relationship (order of relationships matters)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)