You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Charles Robertson <ch...@gmail.com> on 2014/09/10 13:12:58 UTC
Regular expressions in fs paths?
Hi all,
Is it possible to use regular expressions in fs commands? Specifically, I
want to use the copy (-cp) and move (-mv) commands on all files in a
directory that match a pattern (the pattern being all files that do not end
in '.tmp').
Can this be done?
Thanks,
Charles
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
I solved this in the end by using a shell script (initiated by an oozie
shell action) to use grep and loop through the results - didn't have to use
-v option, as the -e option gives you access to a fuller range of regular
expression functionality.
Thanks for your help (again!) Rich.
Charles
On 10 September 2014 16:50, Rich Haase <rd...@gmail.com> wrote:
> HDFS doesn't support he full range of glob matching you will find in
> Linux. If you want to exclude all files from a directory listing that meet
> a certain criteria try doing your listing and using grep -v to exclude the
> matching records.
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
I solved this in the end by using a shell script (initiated by an oozie
shell action) to use grep and loop through the results - didn't have to use
-v option, as the -e option gives you access to a fuller range of regular
expression functionality.
Thanks for your help (again!) Rich.
Charles
On 10 September 2014 16:50, Rich Haase <rd...@gmail.com> wrote:
> HDFS doesn't support he full range of glob matching you will find in
> Linux. If you want to exclude all files from a directory listing that meet
> a certain criteria try doing your listing and using grep -v to exclude the
> matching records.
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
I solved this in the end by using a shell script (initiated by an oozie
shell action) to use grep and loop through the results - didn't have to use
-v option, as the -e option gives you access to a fuller range of regular
expression functionality.
Thanks for your help (again!) Rich.
Charles
On 10 September 2014 16:50, Rich Haase <rd...@gmail.com> wrote:
> HDFS doesn't support he full range of glob matching you will find in
> Linux. If you want to exclude all files from a directory listing that meet
> a certain criteria try doing your listing and using grep -v to exclude the
> matching records.
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
I solved this in the end by using a shell script (initiated by an oozie
shell action) to use grep and loop through the results - didn't have to use
-v option, as the -e option gives you access to a fuller range of regular
expression functionality.
Thanks for your help (again!) Rich.
Charles
On 10 September 2014 16:50, Rich Haase <rd...@gmail.com> wrote:
> HDFS doesn't support he full range of glob matching you will find in
> Linux. If you want to exclude all files from a directory listing that meet
> a certain criteria try doing your listing and using grep -v to exclude the
> matching records.
>
Re: Regular expressions in fs paths?
Posted by Rich Haase <rd...@gmail.com>.
HDFS doesn't support he full range of glob matching you will find in Linux.
If you want to exclude all files from a directory listing that meet a
certain criteria try doing your listing and using grep -v to exclude the
matching records.
Re: Regular expressions in fs paths?
Posted by Rich Haase <rd...@gmail.com>.
HDFS doesn't support he full range of glob matching you will find in Linux.
If you want to exclude all files from a directory listing that meet a
certain criteria try doing your listing and using grep -v to exclude the
matching records.
Re: Regular expressions in fs paths?
Posted by Rich Haase <rd...@gmail.com>.
HDFS doesn't support he full range of glob matching you will find in Linux.
If you want to exclude all files from a directory listing that meet a
certain criteria try doing your listing and using grep -v to exclude the
matching records.
Re: Regular expressions in fs paths?
Posted by Rich Haase <rd...@gmail.com>.
HDFS doesn't support he full range of glob matching you will find in Linux.
If you want to exclude all files from a directory listing that meet a
certain criteria try doing your listing and using grep -v to exclude the
matching records.
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
Hi Georgi,
Thanks for your reply. Won't hadoop fs -ls /tmp/myfiles* return all files
that begin with 'myfiles' in the tmp directory? What I don't understand is
how I can specify a pattern that excludes files ending in '.tmp'. I have
tried using the normal regular expression syntax for this ^(.tmp) but it
tries to match it literally.
Regards,
Charles
On 10 September 2014 13:07, Georgi Ivanov <iv...@vesseltracker.com> wrote:
> Yes you can :
> hadoop fs -ls /tmp/myfiles*
>
> I would recommend first using -ls in order to verify you are selecting
> the right files.
>
> #Mahesh : do you need some help doing this ?
>
>
>
> On 10.09.2014 13:46, Mahesh Khandewal wrote:
>
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
> charles.robertson@gmail.com> wrote:
>
>> Hi all,
>>
>> Is it possible to use regular expressions in fs commands? Specifically,
>> I want to use the copy (-cp) and move (-mv) commands on all files in a
>> directory that match a pattern (the pattern being all files that do not end
>> in '.tmp').
>>
>> Can this be done?
>>
>> Thanks,
>> Charles
>>
>
>
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
Hi Georgi,
Thanks for your reply. Won't hadoop fs -ls /tmp/myfiles* return all files
that begin with 'myfiles' in the tmp directory? What I don't understand is
how I can specify a pattern that excludes files ending in '.tmp'. I have
tried using the normal regular expression syntax for this ^(.tmp) but it
tries to match it literally.
Regards,
Charles
On 10 September 2014 13:07, Georgi Ivanov <iv...@vesseltracker.com> wrote:
> Yes you can :
> hadoop fs -ls /tmp/myfiles*
>
> I would recommend first using -ls in order to verify you are selecting
> the right files.
>
> #Mahesh : do you need some help doing this ?
>
>
>
> On 10.09.2014 13:46, Mahesh Khandewal wrote:
>
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
> charles.robertson@gmail.com> wrote:
>
>> Hi all,
>>
>> Is it possible to use regular expressions in fs commands? Specifically,
>> I want to use the copy (-cp) and move (-mv) commands on all files in a
>> directory that match a pattern (the pattern being all files that do not end
>> in '.tmp').
>>
>> Can this be done?
>>
>> Thanks,
>> Charles
>>
>
>
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
Hi Georgi,
Thanks for your reply. Won't hadoop fs -ls /tmp/myfiles* return all files
that begin with 'myfiles' in the tmp directory? What I don't understand is
how I can specify a pattern that excludes files ending in '.tmp'. I have
tried using the normal regular expression syntax for this ^(.tmp) but it
tries to match it literally.
Regards,
Charles
On 10 September 2014 13:07, Georgi Ivanov <iv...@vesseltracker.com> wrote:
> Yes you can :
> hadoop fs -ls /tmp/myfiles*
>
> I would recommend first using -ls in order to verify you are selecting
> the right files.
>
> #Mahesh : do you need some help doing this ?
>
>
>
> On 10.09.2014 13:46, Mahesh Khandewal wrote:
>
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
> charles.robertson@gmail.com> wrote:
>
>> Hi all,
>>
>> Is it possible to use regular expressions in fs commands? Specifically,
>> I want to use the copy (-cp) and move (-mv) commands on all files in a
>> directory that match a pattern (the pattern being all files that do not end
>> in '.tmp').
>>
>> Can this be done?
>>
>> Thanks,
>> Charles
>>
>
>
>
Re: Regular expressions in fs paths?
Posted by Charles Robertson <ch...@gmail.com>.
Hi Georgi,
Thanks for your reply. Won't hadoop fs -ls /tmp/myfiles* return all files
that begin with 'myfiles' in the tmp directory? What I don't understand is
how I can specify a pattern that excludes files ending in '.tmp'. I have
tried using the normal regular expression syntax for this ^(.tmp) but it
tries to match it literally.
Regards,
Charles
On 10 September 2014 13:07, Georgi Ivanov <iv...@vesseltracker.com> wrote:
> Yes you can :
> hadoop fs -ls /tmp/myfiles*
>
> I would recommend first using -ls in order to verify you are selecting
> the right files.
>
> #Mahesh : do you need some help doing this ?
>
>
>
> On 10.09.2014 13:46, Mahesh Khandewal wrote:
>
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
> charles.robertson@gmail.com> wrote:
>
>> Hi all,
>>
>> Is it possible to use regular expressions in fs commands? Specifically,
>> I want to use the copy (-cp) and move (-mv) commands on all files in a
>> directory that match a pattern (the pattern being all files that do not end
>> in '.tmp').
>>
>> Can this be done?
>>
>> Thanks,
>> Charles
>>
>
>
>
Re: Regular expressions in fs paths?
Posted by Georgi Ivanov <iv...@vesseltracker.com>.
Yes you can :
hadoop fs -ls /tmp/myfiles*
I would recommend first using -ls in order to verify you are selecting
the right files.
#Mahesh : do you need some help doing this ?
On 10.09.2014 13:46, Mahesh Khandewal wrote:
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson
> <charles.robertson@gmail.com <ma...@gmail.com>> wrote:
>
> Hi all,
>
> Is it possible to use regular expressions in fs commands?
> Specifically, I want to use the copy (-cp) and move (-mv) commands
> on all files in a directory that match a pattern (the pattern
> being all files that do not end in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
>
Re: Regular expressions in fs paths?
Posted by Georgi Ivanov <iv...@vesseltracker.com>.
Yes you can :
hadoop fs -ls /tmp/myfiles*
I would recommend first using -ls in order to verify you are selecting
the right files.
#Mahesh : do you need some help doing this ?
On 10.09.2014 13:46, Mahesh Khandewal wrote:
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson
> <charles.robertson@gmail.com <ma...@gmail.com>> wrote:
>
> Hi all,
>
> Is it possible to use regular expressions in fs commands?
> Specifically, I want to use the copy (-cp) and move (-mv) commands
> on all files in a directory that match a pattern (the pattern
> being all files that do not end in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
>
Re: Regular expressions in fs paths?
Posted by Georgi Ivanov <iv...@vesseltracker.com>.
Yes you can :
hadoop fs -ls /tmp/myfiles*
I would recommend first using -ls in order to verify you are selecting
the right files.
#Mahesh : do you need some help doing this ?
On 10.09.2014 13:46, Mahesh Khandewal wrote:
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson
> <charles.robertson@gmail.com <ma...@gmail.com>> wrote:
>
> Hi all,
>
> Is it possible to use regular expressions in fs commands?
> Specifically, I want to use the copy (-cp) and move (-mv) commands
> on all files in a directory that match a pattern (the pattern
> being all files that do not end in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
>
Re: Regular expressions in fs paths?
Posted by Georgi Ivanov <iv...@vesseltracker.com>.
Yes you can :
hadoop fs -ls /tmp/myfiles*
I would recommend first using -ls in order to verify you are selecting
the right files.
#Mahesh : do you need some help doing this ?
On 10.09.2014 13:46, Mahesh Khandewal wrote:
> I want to unsubscribe from this mailing list
>
> On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson
> <charles.robertson@gmail.com <ma...@gmail.com>> wrote:
>
> Hi all,
>
> Is it possible to use regular expressions in fs commands?
> Specifically, I want to use the copy (-cp) and move (-mv) commands
> on all files in a directory that match a pattern (the pattern
> being all files that do not end in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
>
Re: Regular expressions in fs paths?
Posted by Mahesh Khandewal <ma...@gmail.com>.
I want to unsubscribe from this mailing list
On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
charles.robertson@gmail.com> wrote:
> Hi all,
>
> Is it possible to use regular expressions in fs commands? Specifically, I
> want to use the copy (-cp) and move (-mv) commands on all files in a
> directory that match a pattern (the pattern being all files that do not end
> in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
Re: Regular expressions in fs paths?
Posted by Mahesh Khandewal <ma...@gmail.com>.
I want to unsubscribe from this mailing list
On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
charles.robertson@gmail.com> wrote:
> Hi all,
>
> Is it possible to use regular expressions in fs commands? Specifically, I
> want to use the copy (-cp) and move (-mv) commands on all files in a
> directory that match a pattern (the pattern being all files that do not end
> in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
Re: Regular expressions in fs paths?
Posted by Mahesh Khandewal <ma...@gmail.com>.
I want to unsubscribe from this mailing list
On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
charles.robertson@gmail.com> wrote:
> Hi all,
>
> Is it possible to use regular expressions in fs commands? Specifically, I
> want to use the copy (-cp) and move (-mv) commands on all files in a
> directory that match a pattern (the pattern being all files that do not end
> in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>
Re: Regular expressions in fs paths?
Posted by Mahesh Khandewal <ma...@gmail.com>.
I want to unsubscribe from this mailing list
On Wed, Sep 10, 2014 at 4:42 PM, Charles Robertson <
charles.robertson@gmail.com> wrote:
> Hi all,
>
> Is it possible to use regular expressions in fs commands? Specifically, I
> want to use the copy (-cp) and move (-mv) commands on all files in a
> directory that match a pattern (the pattern being all files that do not end
> in '.tmp').
>
> Can this be done?
>
> Thanks,
> Charles
>