You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by hemamoger <he...@gmail.com> on 2018/04/06 06:18:16 UTC
PutHDFS error
<http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/data1.png>
Hi Team,
I am getting error like in the above image. I am not sure wheather PUTHDFS
configuration is proper or not.
Can you pls help with the error. and also what are the confiration i need to
setup for PUTHDFS processor.
Thanks in advance.
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: PutHDFS error
Posted by Sivaprasanna <si...@gmail.com>.
Did you try the approach that Pierre suggested? That should work in all
cases. Can you please share the screenshot of the processor’s configuration
menu?
Also please make sure whatever path value that you have set for the
‘Directory’ property is prefixed with a ‘/‘. Otherwise it will look for the
relative path with respect to the user i.e. /user/nifi/<provided-path>. If
that’s the case, you maybhave to check that directory to see if a file with
the same name already exists.
-
Sivaprasanna
On Fri, 6 Apr 2018 at 12:33 PM, hemamoger <he...@gmail.com> wrote:
> Yes i tried with replace and append that is throwing error that java is not
> supported something like that.
> But i dont have that file in hadoop which i am pushing to hdfs. still its
> throwing that error .
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Re: PutHDFS error
Posted by Sivaprasanna <si...@gmail.com>.
Glad that you have solved it.. If possible can you share how you resolved
it? It might help others who might face this issue in the future.
-
Sivaprasanna
On Fri, 13 Apr 2018 at 7:05 PM, hemamoger <he...@gmail.com> wrote:
> thanks for your suggestions.
> I have got the answer.
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Re: PutHDFS error
Posted by hemamoger <he...@gmail.com>.
thanks for your suggestions.
I have got the answer.
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: PutHDFS error
Posted by Sivaprasanna <si...@gmail.com>.
1. You have shared screenshot for "hdfs dfs -ls" which will basically list
the contents under /user/root since you're executing that as a root but you
have provided "/root" in PutHDFS so please share the list of contents
available under /root i.e. hdfs dfs -ls /root
2. Have you tried Pierre's approach? That should work all the time since it
appends epoch timestamp to the filename which has no possibilities of
having duplicate.
On Mon, Apr 9, 2018 at 10:30 AM, hemamoger <he...@gmail.com> wrote:
> please check the attached images puthdfs_properties.png
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/
> t951/puthdfs_properties.png>
> getsftp_confgrn.png
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/
> t951/getsftp_confgrn.png>
> hadoop_properties.hadoop_properties
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/
> t951/hadoop_properties.hadoop_properties>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Re: PutHDFS error
Posted by hemamoger <he...@gmail.com>.
please check the attached images puthdfs_properties.png
<http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/puthdfs_properties.png>
getsftp_confgrn.png
<http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/getsftp_confgrn.png>
hadoop_properties.hadoop_properties
<http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/hadoop_properties.hadoop_properties>
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: PutHDFS error
Posted by Pierre Villard <pi...@gmail.com>.
Please share your processor configuration and list the content of the
directory where you're pushing your csv files.
2018-04-06 9:03 GMT+02:00 hemamoger <he...@gmail.com>:
> Yes i tried with replace and append that is throwing error that java is not
> supported something like that.
> But i dont have that file in hadoop which i am pushing to hdfs. still its
> throwing that error .
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Re: PutHDFS error
Posted by hemamoger <he...@gmail.com>.
Yes i tried with replace and append that is throwing error that java is not
supported something like that.
But i dont have that file in hadoop which i am pushing to hdfs. still its
throwing that error .
--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Re: PutHDFS error
Posted by Sivaprasanna <si...@gmail.com>.
You can do what Pierre suggested. However, if you don’t want the already
existing file or want to overwrite that with the new file, configure the
PutHDFS processor and set ‘Conflict Resolution Strategy’ to “replace” or
“append” based on your requirement.
-
Sivaprasanna
On Fri, 6 Apr 2018 at 11:57 AM, Pierre Villard <pi...@gmail.com>
wrote:
> Hi,
>
> This means that the name of the file you're pushing in HDFS already exists
> in the target directory.
> Usually people use an UpdateAttribute processor before the PuHDFS to update
> the attribute 'filename' with a timestamped suffix.
> Something like: filename => ${filname}-${now()}
> To be changed according to your needs.
>
> Thanks,
> Pierre
>
>
> 2018-04-06 8:18 GMT+02:00 hemamoger <he...@gmail.com>:
>
> > <
> http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/data1.png
> > >
> >
> >
> >
> > Hi Team,
> >
> > I am getting error like in the above image. I am not sure wheather
> PUTHDFS
> > configuration is proper or not.
> > Can you pls help with the error. and also what are the confiration i need
> > to
> > setup for PUTHDFS processor.
> >
> > Thanks in advance.
> >
> >
> >
> > --
> > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
> >
>
Re: PutHDFS error
Posted by Pierre Villard <pi...@gmail.com>.
Hi,
This means that the name of the file you're pushing in HDFS already exists
in the target directory.
Usually people use an UpdateAttribute processor before the PuHDFS to update
the attribute 'filename' with a timestamped suffix.
Something like: filename => ${filname}-${now()}
To be changed according to your needs.
Thanks,
Pierre
2018-04-06 8:18 GMT+02:00 hemamoger <he...@gmail.com>:
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/t951/data1.png
> >
>
>
>
> Hi Team,
>
> I am getting error like in the above image. I am not sure wheather PUTHDFS
> configuration is proper or not.
> Can you pls help with the error. and also what are the confiration i need
> to
> setup for PUTHDFS processor.
>
> Thanks in advance.
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>