You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by James McMahon <js...@gmail.com> on 2017/04/20 13:35:36 UTC
Re: appending \n to a flow
Can you try something like this Phil:
${uuid:append('\n')}
Jim
On Thu, Apr 20, 2017 at 9:25 AM, <ph...@orange.com> wrote:
> Hello ,
>
> I want to add a “\n “ at the end of my flow ( it’s a record ….in
> fact) before sending the record to a Put HDFS processor in append mode
>
> I tried to put the ReplaceText Processor in the flow with the following
> config
>
>
>
> Search Value (?s)(^.*$)
>
> Replacement Value \\n
>
> Character Set UTF-8
>
> Maximum Buffer Size 1 MB
>
> Replacement Strategy Append
>
> Evaluation Mode Line-by-Line
>
>
>
> following some instructions found here :
>
> https://community.hortonworks.com/questions/48820/apache-
> nifi-adding-attributes-as-newlines-flowfilw.html
>
>
>
> But without any good results it just append “\n” at the end of each
> record and I just got one big line of appended records …separated with //n
>
>
>
>
>
> My SW Env Nifi 1.1.1 Linux Ubuntu Box
>
>
>
> Phil
>
> Best regards
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>
>
RE: appending \n to a flow
Posted by ph...@orange.com.
Thanks Mark ..
I understand the difference of records and flow of records and how to manage them with the “Entire text “ you mention ☺
From: Mark Payne [mailto:markap14@hotmail.com]
Sent: jeudi 20 avril 2017 15:48
To: users@nifi.apache.org
Subject: Re: appending \n to a flow
Phil,
For your configuration of ReplaceText, I think you need a couple of changes. Firstly, you don't
want to append a newline to the end of every line - you want to append it to the end of the FlowFile.
Phil> yes … the issue is that I am calling just before in the Flow, a REST API ( invoke http) that send back a json record , so I just want to add \n record per record
James previous answer works nicely for me .. I will test more deeply ☺
Thx a lot
Phil
So you'd want "Evaluation Mode" set to "Entire Text". Secondly, NiFi is not going to substitute in 'meta characters'
for you here. You'd want to instead type in a literal newline into the value for "Replacement Value" (you can press Shift + Enter/Return)
to insert a new line.
Thanks
-Mark
On Apr 20, 2017, at 9:35 AM, James McMahon <js...@gmail.com>> wrote:
Can you try something like this Phil:
${uuid:append('\n')}
Jim
On Thu, Apr 20, 2017 at 9:25 AM, <ph...@orange.com>> wrote:
Hello ,
I want to add a “\n “ at the end of my flow ( it’s a record ….in fact) before sending the record to a Put HDFS processor in append mode
I tried to put the ReplaceText Processor in the flow with the following config
Search Value (?s)(^.*$)
Replacement Value \\n<smb://n>
Character Set UTF-8
Maximum Buffer Size 1 MB
Replacement Strategy Append
Evaluation Mode Line-by-Line
following some instructions found here :
https://community.hortonworks.com/questions/48820/apache-nifi-adding-attributes-as-newlines-flowfilw.html
But without any good results it just append “\n” at the end of each record and I just got one big line of appended records …separated with //n
My SW Env Nifi 1.1.1 Linux Ubuntu Box
Phil
Best regards
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
Re: appending \n to a flow
Posted by Mark Payne <ma...@hotmail.com>.
Phil,
For your configuration of ReplaceText, I think you need a couple of changes. Firstly, you don't
want to append a newline to the end of every line - you want to append it to the end of the FlowFile.
So you'd want "Evaluation Mode" set to "Entire Text". Secondly, NiFi is not going to substitute in 'meta characters'
for you here. You'd want to instead type in a literal newline into the value for "Replacement Value" (you can press Shift + Enter/Return)
to insert a new line.
Thanks
-Mark
On Apr 20, 2017, at 9:35 AM, James McMahon <js...@gmail.com>> wrote:
Can you try something like this Phil:
${uuid:append('\n')}
Jim
On Thu, Apr 20, 2017 at 9:25 AM, <ph...@orange.com>> wrote:
Hello ,
I want to add a “\n “ at the end of my flow ( it’s a record ….in fact) before sending the record to a Put HDFS processor in append mode
I tried to put the ReplaceText Processor in the flow with the following config
Search Value (?s)(^.*$)
Replacement Value \\n<smb://n>
Character Set UTF-8
Maximum Buffer Size 1 MB
Replacement Strategy Append
Evaluation Mode Line-by-Line
following some instructions found here :
https://community.hortonworks.com/questions/48820/apache-nifi-adding-attributes-as-newlines-flowfilw.html
But without any good results it just append “\n” at the end of each record and I just got one big line of appended records …separated with //n
My SW Env Nifi 1.1.1 Linux Ubuntu Box
Phil
Best regards
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
Re: appending \n to a flow
Posted by James McMahon <js...@gmail.com>.
You're most welcome sir. Cheers! Jim
On Thu, Apr 20, 2017 at 9:44 AM, <ph...@orange.com> wrote:
> J Thanks a lot James it works nicely now ……
>
>
>
> *From:* James McMahon [mailto:jsmcmahon3@gmail.com]
> *Sent:* jeudi 20 avril 2017 15:36
> *To:* users@nifi.apache.org
> *Subject:* Re: appending \n to a flow
>
>
>
> Can you try something like this Phil:
>
>
>
> ${uuid:append('\n')}
>
>
>
> Jim
>
>
>
> On Thu, Apr 20, 2017 at 9:25 AM, <ph...@orange.com> wrote:
>
> Hello ,
>
> I want to add a “\n “ at the end of my flow ( it’s a record ….in
> fact) before sending the record to a Put HDFS processor in append mode
>
> I tried to put the ReplaceText Processor in the flow with the following
> config
>
>
>
> Search Value (?s)(^.*$)
>
> Replacement Value \\n
>
> Character Set UTF-8
>
> Maximum Buffer Size 1 MB
>
> Replacement Strategy Append
>
> Evaluation Mode Line-by-Line
>
>
>
> following some instructions found here :
>
> https://community.hortonworks.com/questions/48820/apache-
> nifi-adding-attributes-as-newlines-flowfilw.html
>
>
>
> But without any good results it just append “\n” at the end of each
> record and I just got one big line of appended records …separated with //n
>
>
>
>
>
> My SW Env Nifi 1.1.1 Linux Ubuntu Box
>
>
>
> Phil
>
> Best regards
>
> _________________________________________________________________________________________________________________________
>
>
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
>
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
>
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
>
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
>
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
>
> they should not be distributed, used or copied without authorisation.
>
> If you have received this email in error, please notify the sender and delete this message and its attachments.
>
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
>
> Thank you.
>
>
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>
>
RE: appending \n to a flow
Posted by ph...@orange.com.
☺ Thanks a lot James it works nicely now ……
From: James McMahon [mailto:jsmcmahon3@gmail.com]
Sent: jeudi 20 avril 2017 15:36
To: users@nifi.apache.org
Subject: Re: appending \n to a flow
Can you try something like this Phil:
${uuid:append('\n')}
Jim
On Thu, Apr 20, 2017 at 9:25 AM, <ph...@orange.com>> wrote:
Hello ,
I want to add a “\n “ at the end of my flow ( it’s a record ….in fact) before sending the record to a Put HDFS processor in append mode
I tried to put the ReplaceText Processor in the flow with the following config
Search Value (?s)(^.*$)
Replacement Value \\n<file:///\\n>
Character Set UTF-8
Maximum Buffer Size 1 MB
Replacement Strategy Append
Evaluation Mode Line-by-Line
following some instructions found here :
https://community.hortonworks.com/questions/48820/apache-nifi-adding-attributes-as-newlines-flowfilw.html
But without any good results it just append “\n” at the end of each record and I just got one big line of appended records …separated with //n
My SW Env Nifi 1.1.1 Linux Ubuntu Box
Phil
Best regards
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.