[jira] [Updated] (METRON-544) Add an integrity checking field for large values

["Jon Zeolla (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METRON-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jon Zeolla updated METRON-544:
------------------------------
    Description: 
If a value is > 32766 long, add an integrity checking field using a hash such as SHA3-256.  This provides a method to both validate and compare the data against another store without a 32k limitation.  For instance, if the field is "uri", add a key of "uri_hash" and a value of the hash of the original field's value (pre-truncation).  

Where this should be added is up for debate - I would initially suggest putting it upstream of the indexingBolt in the indexing topology (as a splitter), but I could see some arguments for putting it in the indexingBolt, or somewhere in the enrichments topology.

  was:
If a value is > 32766 long, add an integrity checking field using a hash such as SHA3-256.  This provides a method to both validate and compare the data against another store without a 32k limitation.  For instance, if the field is "uri", add a key of "uri_hash" and a value of the hash of the original field's value.  

Where this should be added is up for debate - I would initially suggest putting it upstream of the indexingBolt in the indexing topology (as a splitter), but I could see some arguments for putting it in the indexingBolt, or somewhere in the enrichments topology.


> Add an integrity checking field for large values
> ------------------------------------------------
>
>                 Key: METRON-544
>                 URL: https://issues.apache.org/jira/browse/METRON-544
>             Project: Metron
>          Issue Type: Sub-task
>            Reporter: Jon Zeolla
>            Priority: Minor
>
> If a value is > 32766 long, add an integrity checking field using a hash such as SHA3-256.  This provides a method to both validate and compare the data against another store without a 32k limitation.  For instance, if the field is "uri", add a key of "uri_hash" and a value of the hash of the original field's value (pre-truncation).  
> Where this should be added is up for debate - I would initially suggest putting it upstream of the indexingBolt in the indexing topology (as a splitter), but I could see some arguments for putting it in the indexingBolt, or somewhere in the enrichments topology.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)