You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by unmesha sreeveni <un...@gmail.com> on 2015/02/18 08:24:01 UTC
Delete output folder automatically in CRUNCH (FlumeJava)
Hi
I am new to FlumeJava.I ran wordcount in the same.But how can I
automatically delete the outputfolder in the code block. Instead of going
back and deleting the folder.
Thanks in advance.
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks
On Wed, Feb 18, 2015 at 1:44 PM, Suneel Marthi <su...@gmail.com>
wrote:
> Please send the FlumeJava mailing list, this would be better addressed
> there.
>
> On Wed, Feb 18, 2015 at 2:24 AM, unmesha sreeveni <un...@gmail.com>
> wrote:
>
> > Hi
> > I am new to FlumeJava.I ran wordcount in the same.But how can I
> > automatically delete the outputfolder in the code block. Instead of going
> > back and deleting the folder.
> > Thanks in advance.
> >
> > --
> > *Thanks & Regards *
> >
> >
> > *Unmesha Sreeveni U.B*
> > *Hadoop, Bigdata Developer*
> > *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> > http://www.unmeshasreeveni.blogspot.in/
> >
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by Suneel Marthi <su...@gmail.com>.
Please send the FlumeJava mailing list, this would be better addressed
there.
On Wed, Feb 18, 2015 at 2:24 AM, unmesha sreeveni <un...@gmail.com>
wrote:
> Hi
> I am new to FlumeJava.I ran wordcount in the same.But how can I
> automatically delete the outputfolder in the code block. Instead of going
> back and deleting the folder.
> Thanks in advance.
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by unmesha sreeveni <un...@gmail.com>.
sure will do accordingly.
Thanks David.
RE: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by David Ortiz <do...@videologygroup.com>.
Unmesha,
Here’s a better way to go about the output directory deletion.
Pcollection.write(new Target(“/tmp/file”), WriteMode.OVERWRITE);
Using that WriteMode will force it to delete the output directory if it exists.
-Dave
From: unmesha sreeveni [mailto:unmeshabiju@gmail.com]
Sent: Thursday, February 19, 2015 12:02 AM
To: user
Subject: Re: Delete output folder automatically in CRUNCH (FlumeJava)
Thanks David.
This email is intended only for the use of the individual(s) to whom it is addressed. If you have received this communication in error, please immediately notify the sender and delete the original email.
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by unmesha sreeveni <un...@gmail.com>.
Thanks David.
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by David Ortiz <dp...@gmail.com>.
Exactly. That's the object I mean is the hdfs Filesystem. I believe that
would work for both mr and spark implementations.
On Wed, Feb 18, 2015, 11:44 PM unmesha sreeveni <un...@gmail.com>
wrote:
> Is filesystem object similar to mapreduce?
>
> *Configuration conf = new Configuration();*
> *FileSystem fs = FileSystem.get(conf);*
> *if (fs.exists(new Path(args[1]))) {*
> * fs.delete(new Path(args[1]), true);*
> *}*
>
> ** *Is there any good tutorial on Flume Java other than
> https://crunch.apache.org/user-guide.html?
>
> On Thu, Feb 19, 2015 at 10:04 AM, David Ortiz <do...@videologygroup.com>
> wrote:
>
>> You would want to create a Filesystem object in your tool and use it to
>> delete the output directory.
>>
>>
>> Sent from my T-Mobile 4G LTE Device
>>
>>
>> -------- Original message --------
>> From: unmesha sreeveni
>> Date:02/18/2015 11:21 PM (GMT-05:00)
>> To: user@crunch.apache.org
>> Subject: Fwd: Delete output folder automatically in CRUNCH (FlumeJava)
>>
>>
>> Hi
>> I am new to FlumeJava.I ran wordcount in the same.But how can I
>> automatically delete the outputfolder in the code block. Instead of going
>> back and deleting the folder.
>> Thanks in advance.
>> --
>> *Thanks & Regards *
>>
>>
>> *Unmesha Sreeveni U.B *
>> *Hadoop, Bigdata Developer*
>> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
>> http://www.unmeshasreeveni.blogspot.in/
>>
>>
>> *This email is intended only for the use of the individual(s) to whom
>> it is addressed. If you have received this communication in error, please
>> immediately notify the sender and delete the original email.*
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by unmesha sreeveni <un...@gmail.com>.
Is filesystem object similar to mapreduce?
*Configuration conf = new Configuration();*
*FileSystem fs = FileSystem.get(conf);*
*if (fs.exists(new Path(args[1]))) {*
* fs.delete(new Path(args[1]), true);*
*}*
** *Is there any good tutorial on Flume Java other than
https://crunch.apache.org/user-guide.html?
On Thu, Feb 19, 2015 at 10:04 AM, David Ortiz <do...@videologygroup.com>
wrote:
> You would want to create a Filesystem object in your tool and use it to
> delete the output directory.
>
>
> Sent from my T-Mobile 4G LTE Device
>
>
> -------- Original message --------
> From: unmesha sreeveni
> Date:02/18/2015 11:21 PM (GMT-05:00)
> To: user@crunch.apache.org
> Subject: Fwd: Delete output folder automatically in CRUNCH (FlumeJava)
>
>
> Hi
> I am new to FlumeJava.I ran wordcount in the same.But how can I
> automatically delete the outputfolder in the code block. Instead of going
> back and deleting the folder.
> Thanks in advance.
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B *
> *Hadoop, Bigdata Developer*
> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
> *This email is intended only for the use of the individual(s) to whom
> it is addressed. If you have received this communication in error, please
> immediately notify the sender and delete the original email.*
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
RE: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by David Ortiz <do...@videologygroup.com>.
You would want to create a Filesystem object in your tool and use it to delete the output directory.
Sent from my T-Mobile 4G LTE Device
-------- Original message --------
From: unmesha sreeveni
Date:02/18/2015 11:21 PM (GMT-05:00)
To: user@crunch.apache.org
Subject: Fwd: Delete output folder automatically in CRUNCH (FlumeJava)
Hi
I am new to FlumeJava.I ran wordcount in the same.But how can I automatically delete the outputfolder in the code block. Instead of going back and deleting the folder.
Thanks in advance.
--
Thanks & Regards
Unmesha Sreeveni U.B
Hadoop, Bigdata Developer
Centre for Cyber Security | Amrita Vishwa Vidyapeetham
http://www.unmeshasreeveni.blogspot.in/
This email is intended only for the use of the individual(s) to whom it is addressed. If you have received this communication in error, please immediately notify the sender and delete the original email.
Fwd: Delete output folder automatically in CRUNCH (FlumeJava)
Posted by unmesha sreeveni <un...@gmail.com>.
Hi
I am new to FlumeJava.I ran wordcount in the same.But how can I
automatically delete the outputfolder in the code block. Instead of going
back and deleting the folder.
Thanks in advance.
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/