You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Chengi Liu <ch...@gmail.com> on 2013/08/29 01:49:56 UTC
Helper files in python
Hi,
I have four files
mapper.py
mapper_helper.py
reducer.py
reducer_helper.py
How do i specify those helper files in hadoop?
Thanks
Re: Helper files in python
Posted by Shahab Yunus <sh...@gmail.com>.
What you want to do with these? What do you mean by specify? Do you want to
run them as a streaming job?
Have you seen the streaming tutorial for hadoop?
Regards,
Shahab
On Aug 28, 2013 7:50 PM, "Chengi Liu" <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Shahab Yunus <sh...@gmail.com>.
What you want to do with these? What do you mean by specify? Do you want to
run them as a streaming job?
Have you seen the streaming tutorial for hadoop?
Regards,
Shahab
On Aug 28, 2013 7:50 PM, "Chengi Liu" <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Binglin Chang <de...@gmail.com>.
If you mean to make those files available in map/reduce tasks:
How about put them in one directory, say "app", and specify -file app
-mapper app/xxx -reducer app/xx... , hadoop will pack entire dir for you.
On Thu, Aug 29, 2013 at 7:49 AM, Chengi Liu <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Shahab Yunus <sh...@gmail.com>.
What you want to do with these? What do you mean by specify? Do you want to
run them as a streaming job?
Have you seen the streaming tutorial for hadoop?
Regards,
Shahab
On Aug 28, 2013 7:50 PM, "Chengi Liu" <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Binglin Chang <de...@gmail.com>.
If you mean to make those files available in map/reduce tasks:
How about put them in one directory, say "app", and specify -file app
-mapper app/xxx -reducer app/xx... , hadoop will pack entire dir for you.
On Thu, Aug 29, 2013 at 7:49 AM, Chengi Liu <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Binglin Chang <de...@gmail.com>.
If you mean to make those files available in map/reduce tasks:
How about put them in one directory, say "app", and specify -file app
-mapper app/xxx -reducer app/xx... , hadoop will pack entire dir for you.
On Thu, Aug 29, 2013 at 7:49 AM, Chengi Liu <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Binglin Chang <de...@gmail.com>.
If you mean to make those files available in map/reduce tasks:
How about put them in one directory, say "app", and specify -file app
-mapper app/xxx -reducer app/xx... , hadoop will pack entire dir for you.
On Thu, Aug 29, 2013 at 7:49 AM, Chengi Liu <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>
Re: Helper files in python
Posted by Shahab Yunus <sh...@gmail.com>.
What you want to do with these? What do you mean by specify? Do you want to
run them as a streaming job?
Have you seen the streaming tutorial for hadoop?
Regards,
Shahab
On Aug 28, 2013 7:50 PM, "Chengi Liu" <ch...@gmail.com> wrote:
> Hi,
> I have four files
> mapper.py
> mapper_helper.py
> reducer.py
> reducer_helper.py
>
> How do i specify those helper files in hadoop?
> Thanks
>