Query on Using Pig standalone basis

[Somdip <sr...@apple.com>]

Hi All,

Just wanted to know has anyone used Pig on a standalone basis in production environment i.e. without integrating with Haddoop? Is that even a good idea?

How is the performance if we try parsing big files using Pig. Typically we use perl for this but often we run into memory management issues  and Pig seems 
to be much simpler than perl. 

Please share if you have any inputs.

Thanks,
Somdip. 

Re: Query on Using Pig standalone basis

[Ravi Phulari <rp...@yahoo-inc.com>]

Somdip,
I will be surprised to know if anyone is using Pig on a standalone basis in production environment . Pig uses Hadoop's power to process huge data.
HDFS for storing large data and MapReduce for using distributed system for processing huge data.

You can use Pig in standalone mode for learning purpose but using it on a standalone basis in production environment won't be useful.

-
Ravi


On 8/26/10 5:38 PM, "Somdip" <sr...@apple.com> wrote:

Hi All,

Just wanted to know has anyone used Pig on a standalone basis in production environment i.e. without integrating with Haddoop? Is that even a good idea?

How is the performance if we try parsing big files using Pig. Typically we use perl for this but often we run into memory management issues  and Pig seems
to be much simpler than perl.

Please share if you have any inputs.

Thanks,
Somdip.


--