You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by lu...@china-inv.cn on 2018/02/28 12:00:03 UTC
A question to copy files to other folders
Hi, team,
I'm writing a data flow template to copy files from directory A to other
two directories B and C
There are sub directories under A and I need to copy all files under those
sub directories to the same
palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
C/foo/a.txt
I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
creating sub directories under B and C
(i.e. B/foo and C/foo in above example).
The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
attribute of flowfile. But the 'PutFile' doesn't use it
to create.
I don't want to create those folders manually and create data flow by
using 'GetFile' --> 'PutFile' for each folder because there
will be too many data flows.
Is there any processor that support creating folder and copying files?
Thanks
Boying
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
This email message may contain confidential and/or privileged information.
If you are not the intended recipient, please do not read, save, forward,
disclose or copy the contents of this email or open any file attached to
this email. We will be grateful if you could advise the sender immediately
by replying this email, and delete this email and any attachment or links
to this email completely and immediately from your computer system.
答复: Re: Re: A question to copy files to other folders
Posted by lu...@china-inv.cn.
oh, yes. I mixed the two replied emails. sorry :)
发件人:
"Sivaprasanna" <si...@gmail.com>
收件人:
dev@nifi.apache.org
日期:
2018/03/01 13:46
主题:
Re: Re: A question to copy files to other folders
Yep. And that’s what Matt suggested.
On Thu, 1 Mar 2018 at 8:16 AM, <lu...@china-inv.cn> wrote:
> I found that I can set the 'Directory' property of 'PutFile' processor
to
> B/${path} and C/${path}
> to solve this issue. So the patch is not needed I think
>
>
>
>
> 发件人:
> luby@china-inv.cn
> 收件人:
>
> 抄送:
> dev@nifi.apache.org
> 日期:
> 2018/03/01 09:17
> 主题:
> 答复: Re: A question to copy files to other folders
>
>
>
> I haven't tried it yet.
>
> I just find that the PutFile doesn't meet my requirements.
>
> I'll create a patch for PutFile.
>
>
>
> 发件人:
> Matt Burgess <ma...@apache.org>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/28 21:10
> 主题:
> Re: A question to copy files to other folders
>
>
>
> Looking at the code, it appears that PutFile should support the
> creation of arbitrary directories if the Create Missing Directories
> property is set to true. With that and setting the Directory property
> to ${path}, I would think that would create the subdirectories
> properly. If not, what error are you getting?
>
> Regards,
> Matt
>
> On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen <mi...@gmail.com>
> wrote:
> > That seems like a pretty easy thing to fix with PutFile. Could be done
> with
> > a patch to add an attribute that provides a relative path.
> >
> > On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
> >
> >> Hi, team,
> >>
> >> I'm writing a data flow template to copy files from directory A to
> other
> >> two directories B and C
> >>
> >> There are sub directories under A and I need to copy all files under
> those
> >> sub directories to the same
> >> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
> >> C/foo/a.txt
> >>
> >> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't
support
> >> creating sub directories under B and C
> >> (i.e. B/foo and C/foo in above example).
> >>
> >> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
> >> attribute of flowfile. But the 'PutFile' doesn't use it
> >> to create.
> >>
> >> I don't want to create those folders manually and create data flow by
> >> using 'GetFile' --> 'PutFile' for each folder because there
> >> will be too many data flows.
> >>
> >>
> >> Is there any processor that support creating folder and copying
files?
> >>
> >> Thanks
> >>
> >> Boying
> >>
> >>
> >>
> >>
> >> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存
、
> 对外
> >> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告
知
> 发件
> >> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> >>
> >>
> >> This email message may contain confidential and/or privileged
> information.
> >> If you are not the intended recipient, please do not read, save,
> forward,
> >> disclose or copy the contents of this email or open any file attached
> to
> >> this email. We will be grateful if you could advise the sender
> immediately
> >> by replying this email, and delete this email and any attachment or
> links
> >> to this email completely and immediately from your computer system.
> >>
> >>
> >>
> >>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged
information.
>
> If you are not the intended recipient, please do not read, save,
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender
immediately
>
> by replying this email, and delete this email and any attachment or
links
> to this email completely and immediately from your computer system.
>
>
>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对
外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发
件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged
information.
> If you are not the intended recipient, please do not read, save,
forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender
immediately
> by replying this email, and delete this email and any attachment or
links
> to this email completely and immediately from your computer system.
>
>
>
>
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
This email message may contain confidential and/or privileged information.
If you are not the intended recipient, please do not read, save, forward,
disclose or copy the contents of this email or open any file attached to
this email. We will be grateful if you could advise the sender immediately
by replying this email, and delete this email and any attachment or links
to this email completely and immediately from your computer system.
Re: Re: A question to copy files to other folders
Posted by Sivaprasanna <si...@gmail.com>.
Yep. And that’s what Matt suggested.
On Thu, 1 Mar 2018 at 8:16 AM, <lu...@china-inv.cn> wrote:
> I found that I can set the 'Directory' property of 'PutFile' processor to
> B/${path} and C/${path}
> to solve this issue. So the patch is not needed I think
>
>
>
>
> 发件人:
> luby@china-inv.cn
> 收件人:
>
> 抄送:
> dev@nifi.apache.org
> 日期:
> 2018/03/01 09:17
> 主题:
> 答复: Re: A question to copy files to other folders
>
>
>
> I haven't tried it yet.
>
> I just find that the PutFile doesn't meet my requirements.
>
> I'll create a patch for PutFile.
>
>
>
> 发件人:
> Matt Burgess <ma...@apache.org>
> 收件人:
> dev@nifi.apache.org
> 日期:
> 2018/02/28 21:10
> 主题:
> Re: A question to copy files to other folders
>
>
>
> Looking at the code, it appears that PutFile should support the
> creation of arbitrary directories if the Create Missing Directories
> property is set to true. With that and setting the Directory property
> to ${path}, I would think that would create the subdirectories
> properly. If not, what error are you getting?
>
> Regards,
> Matt
>
> On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen <mi...@gmail.com>
> wrote:
> > That seems like a pretty easy thing to fix with PutFile. Could be done
> with
> > a patch to add an attribute that provides a relative path.
> >
> > On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
> >
> >> Hi, team,
> >>
> >> I'm writing a data flow template to copy files from directory A to
> other
> >> two directories B and C
> >>
> >> There are sub directories under A and I need to copy all files under
> those
> >> sub directories to the same
> >> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
> >> C/foo/a.txt
> >>
> >> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
> >> creating sub directories under B and C
> >> (i.e. B/foo and C/foo in above example).
> >>
> >> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
> >> attribute of flowfile. But the 'PutFile' doesn't use it
> >> to create.
> >>
> >> I don't want to create those folders manually and create data flow by
> >> using 'GetFile' --> 'PutFile' for each folder because there
> >> will be too many data flows.
> >>
> >>
> >> Is there any processor that support creating folder and copying files?
> >>
> >> Thanks
> >>
> >> Boying
> >>
> >>
> >>
> >>
> >> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
> 对外
> >> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
> 发件
> >> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
> >>
> >>
> >> This email message may contain confidential and/or privileged
> information.
> >> If you are not the intended recipient, please do not read, save,
> forward,
> >> disclose or copy the contents of this email or open any file attached
> to
> >> this email. We will be grateful if you could advise the sender
> immediately
> >> by replying this email, and delete this email and any attachment or
> links
> >> to this email completely and immediately from your computer system.
> >>
> >>
> >>
> >>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged information.
>
> If you are not the intended recipient, please do not read, save, forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender immediately
>
> by replying this email, and delete this email and any attachment or links
> to this email completely and immediately from your computer system.
>
>
>
>
>
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged information.
> If you are not the intended recipient, please do not read, save, forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender immediately
> by replying this email, and delete this email and any attachment or links
> to this email completely and immediately from your computer system.
>
>
>
>
答复: Re: A question to copy files to other folders
Posted by lu...@china-inv.cn.
I found that I can set the 'Directory' property of 'PutFile' processor to
B/${path} and C/${path}
to solve this issue. So the patch is not needed I think
发件人:
luby@china-inv.cn
收件人:
抄送:
dev@nifi.apache.org
日期:
2018/03/01 09:17
主题:
答复: Re: A question to copy files to other folders
I haven't tried it yet.
I just find that the PutFile doesn't meet my requirements.
I'll create a patch for PutFile.
发件人:
Matt Burgess <ma...@apache.org>
收件人:
dev@nifi.apache.org
日期:
2018/02/28 21:10
主题:
Re: A question to copy files to other folders
Looking at the code, it appears that PutFile should support the
creation of arbitrary directories if the Create Missing Directories
property is set to true. With that and setting the Directory property
to ${path}, I would think that would create the subdirectories
properly. If not, what error are you getting?
Regards,
Matt
On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen <mi...@gmail.com>
wrote:
> That seems like a pretty easy thing to fix with PutFile. Could be done
with
> a patch to add an attribute that provides a relative path.
>
> On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
>
>> Hi, team,
>>
>> I'm writing a data flow template to copy files from directory A to
other
>> two directories B and C
>>
>> There are sub directories under A and I need to copy all files under
those
>> sub directories to the same
>> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
>> C/foo/a.txt
>>
>> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
>> creating sub directories under B and C
>> (i.e. B/foo and C/foo in above example).
>>
>> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
>> attribute of flowfile. But the 'PutFile' doesn't use it
>> to create.
>>
>> I don't want to create those folders manually and create data flow by
>> using 'GetFile' --> 'PutFile' for each folder because there
>> will be too many data flows.
>>
>>
>> Is there any processor that support creating folder and copying files?
>>
>> Thanks
>>
>> Boying
>>
>>
>>
>>
>> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
对外
>> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
发件
>> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>>
>>
>> This email message may contain confidential and/or privileged
information.
>> If you are not the intended recipient, please do not read, save,
forward,
>> disclose or copy the contents of this email or open any file attached
to
>> this email. We will be grateful if you could advise the sender
immediately
>> by replying this email, and delete this email and any attachment or
links
>> to this email completely and immediately from your computer system.
>>
>>
>>
>>
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
This email message may contain confidential and/or privileged information.
If you are not the intended recipient, please do not read, save, forward,
disclose or copy the contents of this email or open any file attached to
this email. We will be grateful if you could advise the sender immediately
by replying this email, and delete this email and any attachment or links
to this email completely and immediately from your computer system.
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
This email message may contain confidential and/or privileged information.
If you are not the intended recipient, please do not read, save, forward,
disclose or copy the contents of this email or open any file attached to
this email. We will be grateful if you could advise the sender immediately
by replying this email, and delete this email and any attachment or links
to this email completely and immediately from your computer system.
答复: Re: A question to copy files to other folders
Posted by lu...@china-inv.cn.
I haven't tried it yet.
I just find that the PutFile doesn't meet my requirements.
I'll create a patch for PutFile.
发件人:
Matt Burgess <ma...@apache.org>
收件人:
dev@nifi.apache.org
日期:
2018/02/28 21:10
主题:
Re: A question to copy files to other folders
Looking at the code, it appears that PutFile should support the
creation of arbitrary directories if the Create Missing Directories
property is set to true. With that and setting the Directory property
to ${path}, I would think that would create the subdirectories
properly. If not, what error are you getting?
Regards,
Matt
On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen <mi...@gmail.com>
wrote:
> That seems like a pretty easy thing to fix with PutFile. Could be done
with
> a patch to add an attribute that provides a relative path.
>
> On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
>
>> Hi, team,
>>
>> I'm writing a data flow template to copy files from directory A to
other
>> two directories B and C
>>
>> There are sub directories under A and I need to copy all files under
those
>> sub directories to the same
>> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
>> C/foo/a.txt
>>
>> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
>> creating sub directories under B and C
>> (i.e. B/foo and C/foo in above example).
>>
>> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
>> attribute of flowfile. But the 'PutFile' doesn't use it
>> to create.
>>
>> I don't want to create those folders manually and create data flow by
>> using 'GetFile' --> 'PutFile' for each folder because there
>> will be too many data flows.
>>
>>
>> Is there any processor that support creating folder and copying files?
>>
>> Thanks
>>
>> Boying
>>
>>
>>
>>
>> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、
对外
>> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知
发件
>> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>>
>>
>> This email message may contain confidential and/or privileged
information.
>> If you are not the intended recipient, please do not read, save,
forward,
>> disclose or copy the contents of this email or open any file attached
to
>> this email. We will be grateful if you could advise the sender
immediately
>> by replying this email, and delete this email and any attachment or
links
>> to this email completely and immediately from your computer system.
>>
>>
>>
>>
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
This email message may contain confidential and/or privileged information.
If you are not the intended recipient, please do not read, save, forward,
disclose or copy the contents of this email or open any file attached to
this email. We will be grateful if you could advise the sender immediately
by replying this email, and delete this email and any attachment or links
to this email completely and immediately from your computer system.
Re: A question to copy files to other folders
Posted by Matt Burgess <ma...@apache.org>.
Looking at the code, it appears that PutFile should support the
creation of arbitrary directories if the Create Missing Directories
property is set to true. With that and setting the Directory property
to ${path}, I would think that would create the subdirectories
properly. If not, what error are you getting?
Regards,
Matt
On Wed, Feb 28, 2018 at 7:12 AM, Mike Thomsen <mi...@gmail.com> wrote:
> That seems like a pretty easy thing to fix with PutFile. Could be done with
> a patch to add an attribute that provides a relative path.
>
> On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
>
>> Hi, team,
>>
>> I'm writing a data flow template to copy files from directory A to other
>> two directories B and C
>>
>> There are sub directories under A and I need to copy all files under those
>> sub directories to the same
>> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
>> C/foo/a.txt
>>
>> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
>> creating sub directories under B and C
>> (i.e. B/foo and C/foo in above example).
>>
>> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
>> attribute of flowfile. But the 'PutFile' doesn't use it
>> to create.
>>
>> I don't want to create those folders manually and create data flow by
>> using 'GetFile' --> 'PutFile' for each folder because there
>> will be too many data flows.
>>
>>
>> Is there any processor that support creating folder and copying files?
>>
>> Thanks
>>
>> Boying
>>
>>
>>
>>
>> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
>> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
>> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>>
>>
>> This email message may contain confidential and/or privileged information.
>> If you are not the intended recipient, please do not read, save, forward,
>> disclose or copy the contents of this email or open any file attached to
>> this email. We will be grateful if you could advise the sender immediately
>> by replying this email, and delete this email and any attachment or links
>> to this email completely and immediately from your computer system.
>>
>>
>>
>>
Re: A question to copy files to other folders
Posted by Mike Thomsen <mi...@gmail.com>.
That seems like a pretty easy thing to fix with PutFile. Could be done with
a patch to add an attribute that provides a relative path.
On Wed, Feb 28, 2018 at 7:00 AM, <lu...@china-inv.cn> wrote:
> Hi, team,
>
> I'm writing a data flow template to copy files from directory A to other
> two directories B and C
>
> There are sub directories under A and I need to copy all files under those
> sub directories to the same
> palce under B and C e.g. copy a file A/foo/a.txt to B/foo/a.txt and
> C/foo/a.txt
>
> I tried processors 'GetFile' and 'PutFile', but PutFile doesn't support
> creating sub directories under B and C
> (i.e. B/foo and C/foo in above example).
>
> The 'GetFile' processor saves the relative path 'foo' in the 'PATH'
> attribute of flowfile. But the 'PutFile' doesn't use it
> to create.
>
> I don't want to create those folders manually and create data flow by
> using 'GetFile' --> 'PutFile' for each folder because there
> will be too many data flows.
>
>
> Is there any processor that support creating folder and copying files?
>
> Thanks
>
> Boying
>
>
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
> 披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
> 人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
> This email message may contain confidential and/or privileged information.
> If you are not the intended recipient, please do not read, save, forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender immediately
> by replying this email, and delete this email and any attachment or links
> to this email completely and immediately from your computer system.
>
>
>
>