You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Jean-Sebastien Vachon <js...@brizodata.com> on 2022/04/11 20:48:01 UTC

Problem restoring value saved to an attribute

Hi all,

I have a fairly complex flow that extract some JSON path to an attribute and then later, will restore this attribute to the flow to form the final Json response.

I made a very simple flow to simulate these operations

[cid:a25e02fb-86ac-45a6-af31-3a442090a55e]

The generated flow file is:

{"content": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

I'm extracting the node content to an attribute using the EvaluateJsonPath with the expression "$.content" and send it to the 'content' attribute.
The two ReplaceText simply restore the saved attribute to the flow with the following configurations

[cid:dcf11c35-cbe8-4dc5-bb81-e9a5e4b6f584][cid:4dc0fd51-6f94-444d-a1c4-c75364d410a9]

The end result I was expecting (and still looking for) is this:

{"body": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

the withEscapeJson configuration returns

{"body": "Coke $1.99nPepsi $1.99nScotch $12.99"}

and the "as is" configuration:

returns

{"body": "Coke $1.99
Pepsi $1.99
Scotch $12.99"}

We can see that the \n characters are preserved correctly but I'm not able to restore them as a valid json construct.

Any idea? I'm pretty sure it will be obvious but I just don't see it.

Thanks


Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>

Re: Problem restoring value saved to an attribute

Posted by Jean-Sebastien Vachon <js...@brizodata.com>.
Hi again,

another update.. I put my fix in place and it seems to work for most flows but I have a few that end up with double escaping

"A LA CARTE\\nDRINKS\\n3 Beef or Chicken Tacos\\n\$8.50\\nTea\\n\$2.85\\nHorchata\\nNO REFILL\\n\$2.85\\n3 Soft Bean Tacos with Chile Con Queso\\n\$8.50\\nCoffee\\n\$2.85\\nOrange Juice NO\\nREFILL\\n\$2.25..."

If I look at the content attribute, it looks exactly like any others (see other example below)

[cid:2a6914bd-6b38-4ef2-8648-f0e4e83aee36]


Any idea why performing a ReplaceText with this expression  "${content:escapeJson()}" would result in double escaping only in some cases?

As an example, this one will end up without the double escaping issue

[cid:d18923a3-3e7c-488c-af68-748c91482ccc]

I don't understand why the behavior is not the same and why the '$' is escaped at all as JSON does not require this char to be escaped.

Thanks


Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>
________________________________
From: Jean-Sebastien Vachon <js...@brizodata.com>
Sent: Monday, April 11, 2022 7:26 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Problem restoring value saved to an attribute

Hi all,

I managed to find a solution but I'd like to know if this is as expected of if there is any other way of fixing this

My solution is to add an UpdateAttribute following the EvaluateJsonPath to properly escape the attribute's value.

[cid:fc40c405-b2bc-4bb9-860b-96364bb45705]

For the record, my EvaluateJsonPath processor is configured as follows

[cid:c9ef9210-f113-481a-bcd9-eff9dd0b4566]

I would have expected the string to be json encoded with this configuration, but it does not seem to be the case.

Thanks

Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>
________________________________
From: Jean-Sebastien Vachon <js...@brizodata.com>
Sent: Monday, April 11, 2022 4:48 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Problem restoring value saved to an attribute

Hi all,

I have a fairly complex flow that extract some JSON path to an attribute and then later, will restore this attribute to the flow to form the final Json response.

I made a very simple flow to simulate these operations

[cid:a25e02fb-86ac-45a6-af31-3a442090a55e]

The generated flow file is:

{"content": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

I'm extracting the node content to an attribute using the EvaluateJsonPath with the expression "$.content" and send it to the 'content' attribute.
The two ReplaceText simply restore the saved attribute to the flow with the following configurations

[cid:dcf11c35-cbe8-4dc5-bb81-e9a5e4b6f584][cid:4dc0fd51-6f94-444d-a1c4-c75364d410a9]

The end result I was expecting (and still looking for) is this:

{"body": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

the withEscapeJson configuration returns

{"body": "Coke $1.99nPepsi $1.99nScotch $12.99"}

and the "as is" configuration:

returns

{"body": "Coke $1.99
Pepsi $1.99
Scotch $12.99"}

We can see that the \n characters are preserved correctly but I'm not able to restore them as a valid json construct.

Any idea? I'm pretty sure it will be obvious but I just don't see it.

Thanks


Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>

Re: Problem restoring value saved to an attribute

Posted by Jean-Sebastien Vachon <js...@brizodata.com>.
Hi all,

I managed to find a solution but I'd like to know if this is as expected of if there is any other way of fixing this

My solution is to add an UpdateAttribute following the EvaluateJsonPath to properly escape the attribute's value.

[cid:fc40c405-b2bc-4bb9-860b-96364bb45705]

For the record, my EvaluateJsonPath processor is configured as follows

[cid:c9ef9210-f113-481a-bcd9-eff9dd0b4566]

I would have expected the string to be json encoded with this configuration, but it does not seem to be the case.

Thanks

Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>
________________________________
From: Jean-Sebastien Vachon <js...@brizodata.com>
Sent: Monday, April 11, 2022 4:48 PM
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Problem restoring value saved to an attribute

Hi all,

I have a fairly complex flow that extract some JSON path to an attribute and then later, will restore this attribute to the flow to form the final Json response.

I made a very simple flow to simulate these operations

[cid:a25e02fb-86ac-45a6-af31-3a442090a55e]

The generated flow file is:

{"content": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

I'm extracting the node content to an attribute using the EvaluateJsonPath with the expression "$.content" and send it to the 'content' attribute.
The two ReplaceText simply restore the saved attribute to the flow with the following configurations

[cid:dcf11c35-cbe8-4dc5-bb81-e9a5e4b6f584][cid:4dc0fd51-6f94-444d-a1c4-c75364d410a9]

The end result I was expecting (and still looking for) is this:

{"body": "Coke $1.99\nPepsi $1.99\nScotch $12.99"}

the withEscapeJson configuration returns

{"body": "Coke $1.99nPepsi $1.99nScotch $12.99"}

and the "as is" configuration:

returns

{"body": "Coke $1.99
Pepsi $1.99
Scotch $12.99"}

We can see that the \n characters are preserved correctly but I'm not able to restore them as a valid json construct.

Any idea? I'm pretty sure it will be obvious but I just don't see it.

Thanks


Jean-Sébastien Vachon
Co-Founder & Architect
Brizo Data, Inc.
www.brizodata.com<https://outlook.office365.com/mail/options/mail/messageContent/www.brizodata.com>