You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by jpconver <jp...@gmail.com> on 2019/04/15 23:23:10 UTC
How to decrypt one field in a Json and return the original JSON
with that field decrypted
Dear all!
I'm a Nifi Newbie and I'd like some guidance to solve this problem.
I have a use case where I receive a JSON with one field encrypted. What I'd
like to do is to process this JSON with NiFi and return the original JSON
but with the field decrypted. I'd like to use the processor EncryptContent
if that's possible.
I know the name of the encrypted field in advance.
For example if a Receive the following json
{"id":"1","name":"paul","password":"encryptedPassword"}
I'd like to return (or have an some point of the flow)
{"id":"1","name":"paul","password":"decryptedPassword"}
What would be the best strategy to achieve this without developing a custom
processor?
Thanks!
JP
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: How to decrypt one field in a Json and return the original JSON
with that field decrypted
Posted by Andy LoPresto <al...@apache.org>.
Here are two Stack Overflow answers [1][2] which should give you a good model to build from (the scripts probably do 90% of what you want right now, and Matt Burgess’ blog [3] on scripting in NiFi should be helpful to modify it as necessary).
[1] https://stackoverflow.com/questions/40294945/nifi-encrypt-json/40295725#40295725 <https://stackoverflow.com/questions/40294945/nifi-encrypt-json/40295725#40295725>
[2] https://stackoverflow.com/a/50536047/70465 <https://stackoverflow.com/a/50536047/70465>
[3] https://funnifi.blogspot.com/ <https://funnifi.blogspot.com/>
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Apr 16, 2019, at 9:31 AM, Andy LoPresto <al...@apache.org> wrote:
>
> Hi JP,
>
> I do plan to implement an EncryptAttribute and EncryptRecord processor in the near future; other deliverables have taken priority recently. My suggestion for this with the least amount of complexity (but some custom code generation) would be to use ExecuteScript with simple Groovy code to leverage the Java Cryptographic Extension services that EncryptContent would use anyway.
>
> I can provide a proof of concept implementation over the next couple days (I’m traveling right now) that does this.
>
> Another approach would be to use the EvaluateJSONPath to extract the encrypted password from JSON to an attribute, perform the same decryption logic using ExecuteScript but on a specific attribute rather than parse the JSON in the ExecuteScript processor, and then replace the decrypted value into the flowfile content with ReplaceText.
>
> Sorry there is not an out-of-the-box solution for you at this time.
>
>
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> alopresto.apache@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>> On Apr 16, 2019, at 7:26 AM, Otto Fowler <ottobackwards@gmail.com <ma...@gmail.com>> wrote:
>>
>> DecryptRecord ( with record path setting ) sounds like it could be
>> something that could do this if it existed.
>>
>>
>> On April 16, 2019 at 08:49:41, Peter Turcsanyi (
>> turcsanyi@cloudera.com.invalid <ma...@cloudera.com.invalid>) wrote:
>>
>> Hi JP,
>>
>> If I understand correctly, your scenario would be:
>> 1. extract the encrypted data item from json (into a flowfile attribute)
>> 2. decrypt the data
>> 3. replace the encrypted data item with the decrypted one in json
>>
>> Unfortunately, there is no EncryptAttribute processor at the moment which
>> would be more suitable for this scenario (though there is an open issue
>> <https://issues.apache.org/jira/browse/NIFI-2961 <https://issues.apache.org/jira/browse/NIFI-2961>> for it).
>>
>> You can do it with EncryptContent too but it is a bit complicated because
>> you need to put the encrypted data into the flowfile content and decrypt it
>> there.
>>
>> A possible scenario:
>> - back up the original json into an attribute** with ExtractText
>> - extract the encrypted data item from the json into an attribute with
>> EvaluateJsonPath
>> - replace the whole flowfile content to the encrypted data with ReplaceText
>> - base64 decode the encrypted data in the flowfile content with
>> Base64EncodeContent (I supposed your json contains the password as base64
>> encoded string)
>> - decrypt the flowfile content with EncryptContent
>> - copy the decrypted password from the content into an attribute with
>> ExtractText
>> - restore the original json into the content (from the attribute where it
>> was saved) with ReplaceText
>> - replace the encrypted data item with the decrypted one in the json with
>> ReplaceText
>>
>> **The drawback is (beyond the complexity) that the flowfile content (the
>> original json) needs to be stored in an attribute which is not really
>> recommended (or only in case of small files).
>>
>> It could be avoided by splitting the flow into 2 branches, one for the
>> original json (in the conient, not in an attribute) and one for the data
>> decryption, then merge the two branches with MergeContent. However, it
>> would be more complicated than the previous one because you need to handle
>> the merging of the two branches.
>>
>> If there won't be better suggestions to solve your scenario, I can share
>> the sample flow I described above.
>>
>>
>> Regards,
>> Peter
>>
>> On Tue, Apr 16, 2019 at 1:34 AM jpconver <jpconver@gmail.com <ma...@gmail.com>> wrote:
>>
>>> Dear all!
>>>
>>> I'm a Nifi Newbie and I'd like some guidance to solve this problem.
>>>
>>> I have a use case where I receive a JSON with one field encrypted. What
>> I'd
>>> like to do is to process this JSON with NiFi and return the original JSON
>>> but with the field decrypted. I'd like to use the processor
>> EncryptContent
>>> if that's possible.
>>> I know the name of the encrypted field in advance.
>>>
>>> For example if a Receive the following json
>>>
>>> {"id":"1","name":"paul","password":"encryptedPassword"}
>>>
>>> I'd like to return (or have an some point of the flow)
>>>
>>> {"id":"1","name":"paul","password":"decryptedPassword"}
>>>
>>> What would be the best strategy to achieve this without developing a
>> custom
>>> processor?
>>>
>>> Thanks!
>>> JP
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/ <http://apache-nifi-developer-list.39713.n7.nabble.com/>
>>>
>
Re: How to decrypt one field in a Json and return the original JSON
with that field decrypted
Posted by Andy LoPresto <al...@apache.org>.
Hi JP,
I do plan to implement an EncryptAttribute and EncryptRecord processor in the near future; other deliverables have taken priority recently. My suggestion for this with the least amount of complexity (but some custom code generation) would be to use ExecuteScript with simple Groovy code to leverage the Java Cryptographic Extension services that EncryptContent would use anyway.
I can provide a proof of concept implementation over the next couple days (I’m traveling right now) that does this.
Another approach would be to use the EvaluateJSONPath to extract the encrypted password from JSON to an attribute, perform the same decryption logic using ExecuteScript but on a specific attribute rather than parse the JSON in the ExecuteScript processor, and then replace the decrypted value into the flowfile content with ReplaceText.
Sorry there is not an out-of-the-box solution for you at this time.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Apr 16, 2019, at 7:26 AM, Otto Fowler <ot...@gmail.com> wrote:
>
> DecryptRecord ( with record path setting ) sounds like it could be
> something that could do this if it existed.
>
>
> On April 16, 2019 at 08:49:41, Peter Turcsanyi (
> turcsanyi@cloudera.com.invalid) wrote:
>
> Hi JP,
>
> If I understand correctly, your scenario would be:
> 1. extract the encrypted data item from json (into a flowfile attribute)
> 2. decrypt the data
> 3. replace the encrypted data item with the decrypted one in json
>
> Unfortunately, there is no EncryptAttribute processor at the moment which
> would be more suitable for this scenario (though there is an open issue
> <https://issues.apache.org/jira/browse/NIFI-2961> for it).
>
> You can do it with EncryptContent too but it is a bit complicated because
> you need to put the encrypted data into the flowfile content and decrypt it
> there.
>
> A possible scenario:
> - back up the original json into an attribute** with ExtractText
> - extract the encrypted data item from the json into an attribute with
> EvaluateJsonPath
> - replace the whole flowfile content to the encrypted data with ReplaceText
> - base64 decode the encrypted data in the flowfile content with
> Base64EncodeContent (I supposed your json contains the password as base64
> encoded string)
> - decrypt the flowfile content with EncryptContent
> - copy the decrypted password from the content into an attribute with
> ExtractText
> - restore the original json into the content (from the attribute where it
> was saved) with ReplaceText
> - replace the encrypted data item with the decrypted one in the json with
> ReplaceText
>
> **The drawback is (beyond the complexity) that the flowfile content (the
> original json) needs to be stored in an attribute which is not really
> recommended (or only in case of small files).
>
> It could be avoided by splitting the flow into 2 branches, one for the
> original json (in the conient, not in an attribute) and one for the data
> decryption, then merge the two branches with MergeContent. However, it
> would be more complicated than the previous one because you need to handle
> the merging of the two branches.
>
> If there won't be better suggestions to solve your scenario, I can share
> the sample flow I described above.
>
>
> Regards,
> Peter
>
> On Tue, Apr 16, 2019 at 1:34 AM jpconver <jp...@gmail.com> wrote:
>
>> Dear all!
>>
>> I'm a Nifi Newbie and I'd like some guidance to solve this problem.
>>
>> I have a use case where I receive a JSON with one field encrypted. What
> I'd
>> like to do is to process this JSON with NiFi and return the original JSON
>> but with the field decrypted. I'd like to use the processor
> EncryptContent
>> if that's possible.
>> I know the name of the encrypted field in advance.
>>
>> For example if a Receive the following json
>>
>> {"id":"1","name":"paul","password":"encryptedPassword"}
>>
>> I'd like to return (or have an some point of the flow)
>>
>> {"id":"1","name":"paul","password":"decryptedPassword"}
>>
>> What would be the best strategy to achieve this without developing a
> custom
>> processor?
>>
>> Thanks!
>> JP
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>>
Re: How to decrypt one field in a Json and return the original JSON
with that field decrypted
Posted by Otto Fowler <ot...@gmail.com>.
DecryptRecord ( with record path setting ) sounds like it could be
something that could do this if it existed.
On April 16, 2019 at 08:49:41, Peter Turcsanyi (
turcsanyi@cloudera.com.invalid) wrote:
Hi JP,
If I understand correctly, your scenario would be:
1. extract the encrypted data item from json (into a flowfile attribute)
2. decrypt the data
3. replace the encrypted data item with the decrypted one in json
Unfortunately, there is no EncryptAttribute processor at the moment which
would be more suitable for this scenario (though there is an open issue
<https://issues.apache.org/jira/browse/NIFI-2961> for it).
You can do it with EncryptContent too but it is a bit complicated because
you need to put the encrypted data into the flowfile content and decrypt it
there.
A possible scenario:
- back up the original json into an attribute** with ExtractText
- extract the encrypted data item from the json into an attribute with
EvaluateJsonPath
- replace the whole flowfile content to the encrypted data with ReplaceText
- base64 decode the encrypted data in the flowfile content with
Base64EncodeContent (I supposed your json contains the password as base64
encoded string)
- decrypt the flowfile content with EncryptContent
- copy the decrypted password from the content into an attribute with
ExtractText
- restore the original json into the content (from the attribute where it
was saved) with ReplaceText
- replace the encrypted data item with the decrypted one in the json with
ReplaceText
**The drawback is (beyond the complexity) that the flowfile content (the
original json) needs to be stored in an attribute which is not really
recommended (or only in case of small files).
It could be avoided by splitting the flow into 2 branches, one for the
original json (in the conient, not in an attribute) and one for the data
decryption, then merge the two branches with MergeContent. However, it
would be more complicated than the previous one because you need to handle
the merging of the two branches.
If there won't be better suggestions to solve your scenario, I can share
the sample flow I described above.
Regards,
Peter
On Tue, Apr 16, 2019 at 1:34 AM jpconver <jp...@gmail.com> wrote:
> Dear all!
>
> I'm a Nifi Newbie and I'd like some guidance to solve this problem.
>
> I have a use case where I receive a JSON with one field encrypted. What
I'd
> like to do is to process this JSON with NiFi and return the original JSON
> but with the field decrypted. I'd like to use the processor
EncryptContent
> if that's possible.
> I know the name of the encrypted field in advance.
>
> For example if a Receive the following json
>
> {"id":"1","name":"paul","password":"encryptedPassword"}
>
> I'd like to return (or have an some point of the flow)
>
> {"id":"1","name":"paul","password":"decryptedPassword"}
>
> What would be the best strategy to achieve this without developing a
custom
> processor?
>
> Thanks!
> JP
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Re: How to decrypt one field in a Json and return the original JSON
with that field decrypted
Posted by Peter Turcsanyi <tu...@cloudera.com.INVALID>.
Hi JP,
If I understand correctly, your scenario would be:
1. extract the encrypted data item from json (into a flowfile attribute)
2. decrypt the data
3. replace the encrypted data item with the decrypted one in json
Unfortunately, there is no EncryptAttribute processor at the moment which
would be more suitable for this scenario (though there is an open issue
<https://issues.apache.org/jira/browse/NIFI-2961> for it).
You can do it with EncryptContent too but it is a bit complicated because
you need to put the encrypted data into the flowfile content and decrypt it
there.
A possible scenario:
- back up the original json into an attribute** with ExtractText
- extract the encrypted data item from the json into an attribute with
EvaluateJsonPath
- replace the whole flowfile content to the encrypted data with ReplaceText
- base64 decode the encrypted data in the flowfile content with
Base64EncodeContent (I supposed your json contains the password as base64
encoded string)
- decrypt the flowfile content with EncryptContent
- copy the decrypted password from the content into an attribute with
ExtractText
- restore the original json into the content (from the attribute where it
was saved) with ReplaceText
- replace the encrypted data item with the decrypted one in the json with
ReplaceText
**The drawback is (beyond the complexity) that the flowfile content (the
original json) needs to be stored in an attribute which is not really
recommended (or only in case of small files).
It could be avoided by splitting the flow into 2 branches, one for the
original json (in the conient, not in an attribute) and one for the data
decryption, then merge the two branches with MergeContent. However, it
would be more complicated than the previous one because you need to handle
the merging of the two branches.
If there won't be better suggestions to solve your scenario, I can share
the sample flow I described above.
Regards,
Peter
On Tue, Apr 16, 2019 at 1:34 AM jpconver <jp...@gmail.com> wrote:
> Dear all!
>
> I'm a Nifi Newbie and I'd like some guidance to solve this problem.
>
> I have a use case where I receive a JSON with one field encrypted. What I'd
> like to do is to process this JSON with NiFi and return the original JSON
> but with the field decrypted. I'd like to use the processor EncryptContent
> if that's possible.
> I know the name of the encrypted field in advance.
>
> For example if a Receive the following json
>
> {"id":"1","name":"paul","password":"encryptedPassword"}
>
> I'd like to return (or have an some point of the flow)
>
> {"id":"1","name":"paul","password":"decryptedPassword"}
>
> What would be the best strategy to achieve this without developing a custom
> processor?
>
> Thanks!
> JP
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>