You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by zhang jianfeng <zj...@gmail.com> on 2009/06/22 07:47:42 UTC

How to make the pig unsplittable ?

Hi all,

Because of my input file format, the first line of file is the definition of
each field, and then lines of records. So I did not found one good method of
using customer slicer.

So I'd like to make the pig do not split my file, but I did not found an
easy way. Now I have to change the code in POLoad and LOLoad, make the
variable isSplitable false.

Is there any easier way to make it unsplittable, such as configuration ?


Thank you for any help.


Jeff Zhang

Re: How to make the pig unsplittable ?

Posted by zhang jianfeng <zj...@gmail.com>.
Pradeep

Thank you very much

Jeff Zhang



On Tue, Jun 23, 2009 at 2:04 AM, Pradeep Kamath <pr...@yahoo-inc.com>wrote:

> You can load using the following syntax:
> A = load 'inputfile' split by 'file';
>
> The "split by 'file'" will ensure issplittable is set to false and the
> input is not split.
>
> -Pradeep
>
> -----Original Message-----
> From: zhang jianfeng [mailto:zjffdu@gmail.com]
> Sent: Sunday, June 21, 2009 10:48 PM
> To: pig-user@hadoop.apache.org
> Subject: How to make the pig unsplittable ?
>
> Hi all,
>
> Because of my input file format, the first line of file is the
> definition of
> each field, and then lines of records. So I did not found one good
> method of
> using customer slicer.
>
> So I'd like to make the pig do not split my file, but I did not found an
> easy way. Now I have to change the code in POLoad and LOLoad, make the
> variable isSplitable false.
>
> Is there any easier way to make it unsplittable, such as configuration ?
>
>
> Thank you for any help.
>
>
> Jeff Zhang
>

RE: How to make the pig unsplittable ?

Posted by Pradeep Kamath <pr...@yahoo-inc.com>.
You can load using the following syntax:
A = load 'inputfile' split by 'file';

The "split by 'file'" will ensure issplittable is set to false and the
input is not split.

-Pradeep

-----Original Message-----
From: zhang jianfeng [mailto:zjffdu@gmail.com] 
Sent: Sunday, June 21, 2009 10:48 PM
To: pig-user@hadoop.apache.org
Subject: How to make the pig unsplittable ?

Hi all,

Because of my input file format, the first line of file is the
definition of
each field, and then lines of records. So I did not found one good
method of
using customer slicer.

So I'd like to make the pig do not split my file, but I did not found an
easy way. Now I have to change the code in POLoad and LOLoad, make the
variable isSplitable false.

Is there any easier way to make it unsplittable, such as configuration ?


Thank you for any help.


Jeff Zhang